If you listen to, well, pretty much anyone rational, they will tell you in no uncertain terms that the last thing you ever want to do is put your SQL Database on the public internet. Even if you're crazy enough to do that, you certainly should never post the address to it on a place like Hacker News. We did it anyway, and this post describes why we did it, what we learned and what people tried to do with it.
In the world of databases, benchmarking performance has always been the hottest topic. Who is faster for data ingestion and queries? About a month ago we announced a new release with SIMD aggregations on HackerNews and Reddit. Fast. But were those results numerically accurate?
SIMD instructions are specific CPU instruction sets for arithmetic calculations that use synthetic parallelization. This approach allows us to perform the same calculations and operations on numerous data points simultaneously. This post describes how SIMD works with typical operation performance and describes additional optimizations we managed to achieve.
Inter-thread messaging is a fundamental part of any asynchronous system. It is the component responsible for the transportation of data between threads. Messaging forms the infrastructure, scaffolding multi-threaded applications, and just like real-world transport infrastructure, we want it to be inexpensive, fast, reliable, and clean. For QuestDB, we wrote our own messaging system, and this post is about how it works and how fast it is.
InfluxDB is the current market leader in time series. This post examines their ingestion format called InfluxDB line protocol ("ILP") and compares data ingestion performance between QuestDB and InfluxDB. We'll look at data loss over UDP and some of the reasons why QuestDB is more efficient at ingesting records in ILP.