Redpanda is an open-source, Kafka-compatible streaming platform that uses C++ and Raft to replace Java and Zookeeper. Since it is Kafka compatible, it can be used with the QuestDB Kafka connector, providing an alternative data streaming option.
- A local JDK installation
- A running QuestDB instance
#Configure and start Redpanda
Quick start guide
docker-compose.yaml file that can be used. Copy and paste the
content into into a file named
docker-compose.yml on your local filesystem:
Once the file is saved, run the following command to start a single Redpanda broker inside Docker and expose Redpanda to your host machine:
It also start the Redpanda web UI.
#Download Apache Kafka
Download Apache Kafka and unzip the file.
This step is required as Redpanda does not have its own Kafka Connect equivalent.
#Download the QuestDB Kafka connector
the QuestDB Kafka connector,
under the zip archive named
You can automate downloading the latest connector package by running this command:
Unzip the connector - it has a directory with 2 JARs: Copy these JARs into /path/to/kafka/lib:
There should be already a lot of other JAR files. That's how you can tell you are in the right directory.
Go to /path/to/kafka/config - there should be already quite a few *.property
files. Create a new file:
questdb-connector.properties with the following
In addition, pointing the open
connect-standalone.properties and replace:
with the Redpanda broker URL:
#Start Kafka Connect
Navigate to the Kafka Connect folder and then run:
Now the Kafka Connect is initiated.
#Send a message
Open the Redpand UI topic page,
http://127.0.0.1:8080/topics.It should display
If the topic is not there then refresh a few times.
example-topic to expand more details and click
Paste the following message into the message box:
Then, click 'Publish'.
#See result from QuestDB
Go to QuestDB web console at
http://localhost:9000/. Run a
The message is delivered to QuestDB: