Use case: Sensor data and IoT
Industry: Engineering, manufactoring
Deployment: QuestDB Open Source
DATRON builds high-speed industrial CNC machines which output high-frequency sensor data requiring massive ingestion rates at the data layer.
In this case study, we summarize the experience of DATRON evaluating a database that can handle high-ingestion rates for operational integrity and showcase the improvements they saw.
The CNC milling machines that DATRON builds must function reliably and continuously for decades. Medical applications and the aerospace sector require that the mechanical components are manufactured precisely each time with exceptionally low tolerances.
DATRON rigorously tests their installed assemblies in long-term studies over several months, which detect unusual vibrations, wear, and temperature deviations. The metrics collected are the environmental temperature alongside the machine's spindle RPM, while an accelerometer measures the spindle position in three axes. An Arduino polls the spindle sensors at a frequency in the two-digit kHz range, meaning that the testing phase generates multiple terabytes of data. Storing these measurements requires a stable, high-performance database that can write and query time series data efficiently.
Initially, DATRON was experimenting with InfluxDB as a datastore but immediately encountered problems ingesting high-throughput workloads. A recurring issue that surfaced was InfluxDB consuming too much system RAM and an inability to keep up with ingestion rates, causing frequent memory cache flushing. QuestDB was a capable drop-in replacement in this scenario and allowed DATRON to do more with less hardware.
DATRON uses the QuestDB Docker image, which is a turnkey, out-of-the-box solution. Using ILP (InfluxDB line protocol), the data is inserted and read out via REST API to other software for analysis. Inserting and reading the information is possible with popular high-level protocols and little programming effort, a definite advantage over hand-crafted or bespoke solutions.
The developer and community support mean that questions get answered quicker, leading to faster integration and troubleshooting through shorter feedback loops. Using Slack as a communication tool is an additional convenience that bridges the gap between ideas and implementation and adds to the agility of the project to deliver a feature addition or bug fix.
“QuestDB offers new possibilities while reducing costs and simplifying data analysis at DATRON.”
Tim Borowski