Redpanda is an open-source, Kafka-compatible streaming platform that uses C++ and Raft to replace Java and Zookeeper. Since it is Kafka compatible, it can be used with the QuestDB Kafka connector, providing an alternative data streaming option.
- A local JDK installation
- A running QuestDB instance
Quick start guide
docker-compose.yaml file that can be used. Copy and paste the
content into into a file named
docker-compose.yml on your local filesystem:
Once the file is saved, run the following command to start a single Redpanda broker inside Docker and expose Redpanda to your host machine:
It also start the Redpanda web UI.
Download Apache Kafka and unzip the file.
This step is required as Redpanda does not have its own Kafka Connect equivalent.
the QuestDB Kafka connector,
under the zip archive named
You can automate downloading the latest connector package by running this command:
Unzip the connector - it has a directory with 2 JARs: Copy these JARs into /path/to/kafka/lib:
There should be already a lot of other JAR files. That's how you can tell you are in the right directory.
Go to /path/to/kafka/config - there should be already quite a few *.property
files. Create a new file:
questdb-connector.properties with the following
In addition, pointing the open
connect-standalone.properties and replace:
with the Redpanda broker URL:
Navigate to the Kafka Connect folder and then run:
Now the Kafka Connect is initiated.
Open the Redpand UI topic page,
http://127.0.0.1:8080/topics.It should display
If the topic is not there then refresh a few times.
example-topic to expand more details and click
Paste the following message into the message box:
Then, click 'Publish'.
Go to QuestDB web console at
http://localhost:9000/. Run a
The message is delivered to QuestDB: