This tutorial covers my first steps of connecting an IoT device to QuestDB to explore the features and functionality. After using InfluxDB as a main data store for sensor data in the past, I'll be looking at the workflow and ease of use from this perspective and showing how simple it is for me to make the switch over to QuestDB.
This tutorial uses the UDP receiver, which is deprecated since QuestDB version 6.5.2. We recommend the TCP receiver instead.
#The Database Part
The first thing I had to do was to get QuestDB up and running. Luckily, this is
very straightforward. There are a few options, the
Docker image for those who want to run without
installing any dependencies, a Homebrew install
brew install questdb, grabbing the binaries
directly, and building from source. Since I work here, and I wanted to test out
the latest and greatest Web Console, I decided to build using maven:
It builds really quickly due to the lack of external dependencies, so that is great! Then all that's left to do is to start the server:
That is literally all there is to getting QuestDB built and running. But that's just the first part. Now it's time to do something actually useful with it. First, I'll need to create a table in QuestDB to store my IoT Data (A bit more on this later, so store a pointer to this).
Remember, we're using standard ANSI SQL here, so there's no new language or syntax to learn. This is a straightforward table that I'm building because I'm going to be using an ESP8266 with an (honestly not very accurate) DHT11 temperature and humidity sensor on it.
#The Sensor Part
For this, I'm going to use an ESP8266-based WEMOS D1 Mini only because I happen to have a giant pile of them lying around. I buy them in bulk because they are a dollar or 2 each, easy to use, and largely disposable if I blow one up (which I do with alarming regularity.). The circuit is very simple to put together:
I used an actual WEMOS Shield with the DHT11 on it, so I didn't have to breadboard it, but this schematic gives you an idea of how simple the wiring is. It's literally 3 wires.
#The Code Part
Here is where the magic happens. How I actually send the sensor data to the database. There is a simple example program included with the Adafruit DHT Unified Sensor Library that I recommend starting with to make this a bit easier. It already has all the parts to read from the sensor, so you don't have to write those from scratch. Remember, good developers copy, but great developers paste!
Since I'm using the 8266, and I'll need internet connectivity, I'll need all the WiFi bits:
Really all you have to do is go to the 'Sketch' Menu, Choose 'Include Library' and select the 'ESP8266WiFi' library, and you get all this stuff imported for you.
Here's some boilerplate code you can always use to get your ESP8266 on your WiFi:
That gets the basics set up. Running that should get you a WiFi connection and a fully configured DHT11 sensor. We're almost ready to start sending data to the database.
You might have noticed the UDP stuff I snuck in there. That's because we're going to make this super easy and use UDP to send our data. And there's a really good reason for that: InfluxDB line protocol. You see, QuestDB has a built-in InfluxDB line Protocol listener, but (for now) it's only listening on a UDP port. So we're going to use that.
Now, to send some data:
Yeah, a lot is going on in there, so let's break it down. First, I'm creating a
buffer to hold the data I'm going to send, and then I'll do a read of the
sensor. I set the
hum variables to values that I know the sensor
will never return so that I can check that I got valid readings later to avoid
sending gibberish to the database.
I have to do some shenanigans with the temperature and humidity values in there
because one of the shortcomings of Arduinos is that they don't have
support for doubles. I know. So I turn them into strings and move on. Once they
arrive at the database, they are interpreted as doubles and life is good. Not
worth fighting about. I can then construct a buffer with straight line protocol
and ship it off to QuestDB over UDP.
Don't forget to free the memory!
Remember I told you to set a pointer earlier about creating the database? Well, here's where I come back to that. You don't actually have to create the database ahead of time unless you want to do things like set indexes, etc. If all you want to do is have straight values in there, then guess what? Schema-on-write is a thing here. You can start writing data to the database, and it will happily store them for you. Pretty cool stuff.
#Querying the Data
Using the QuestDB Console, you can then query the data to make sure you're getting what you expect:
That's exactly what I expected!
Now it's time to start building some dashboards on top of this. I'm currently working on connecting this all up with Node-Red, or making use of the support for Grafana, which will be huge, so stay tuned for that. If you like what you see here, pleases go give us a star on GitHub, and follow the project if you'd like to get updates!