QuestDB 8.0.3 - JSON support, smarter Web Console, and more

Summer time! 🌞 For some, that's sunshine, beaches, and lotion, all that fun stuff. Or, perhaps, hiding inside near the air conditioner because it's hot! For QuestDB, it's another set of improvements. This release contains a bundle of smaller improvements, with new features laying down major groundwork. And - of course - the usual motion of performance improvements. Hot indeed. 🔥

Upgrade! Apply liberal amounts of sunscreen!

For the full notes including PR links, checkout GitHub.

Breaking changes 💥

• QuestDB no longer supports FreeBSD. We test major browser versions on each release. That means everything we write needs to account for multiple OS paradigms. FreeBSD has very small market share, and more often than not we'd trip over edge cases. To speed things up, we've made the decision to drop support. This means FreeBSD is no longer tested during development.

• NFS is now an unsupported file system. QuestDB will refuse to start if running on an NFS disk. We recommend using ZFS. For a full list of supported filesystems, checkout the Capacity Planning documentation.

New features 🐣

JSON support

The json_extract() function is the first step towards robust JSON support. Leveraging the simdjson library by Daniel Lemire, it allows extracting fields from a JSON document and storing them into VARCHAR columns. There is some performance overhead, so we recommend using it to test a schema before moving from JSON to native column types.

SELECT
    json_extract(trade_details, '$.quantity')::long quantity,
    json_extract(trade_details, '$.price')::double price,
    json_extract(trade_details, '$.executions[0].timestamp')::timestamp first_ex_ts
FROM
    trades
WHERE
    json_extract(trade_details, '$.exchange') = 'NASDAQ'

Read more in the documentation.

New financial functions

Introducing the mid_price function and spread function for financial calculations. Simple, but effective enhancements for financial data processing.

Smarter Web Console

Vague errors aren't helpful. The Web Console now helps you self-troubleshoot common database configuration issues with clear visual indicators to identify and resolve problems.

For example, what if one of our tables is suspended?

The UI will clearly show there's an issue with the table. There is an issue with trips. Knowing there's an issue is half of it. But now, what can we do?

In many cases, we can identify the specific issue and offer specific guidance to help resolve it. In this case, there is no disk space. There's a link to the docs to help, and the ability to restart once you've resolved the root issue.

Look forward to more improvements in this way as we get better at providing clear erroring and resoluton steps.

HTTP Basic Auth

QuestDB open source now offers Basic Auth to protect its HTTP endpoints.

See the documentation for configuration guidance.

Performance 🚀

As always, QuestDB is a little faster, better, and stronger than before:

• JIT-optimized IN(numeric values) operator for faster query execution.

• Optimized aggregate queries with the LAST function on designated timestamps.

• Speed improvements for self-join queries on symbol columns and more efficient pattern lookups for short ASCII patterns.

• Enhancements to GROUP BY operations and parallel execution, particularly benefiting queries with aggregate functions like avg() and sum().

Bug fixes 🐛

And we've squashed some pesky bugs:

• VARCHAR column queries via sqlx::postgres.
• Invalid column errors in GROUP BY with JOIN.
• Improved error handling to prevent ugly error messages when the log directory does not exist.
• Ensuring REST interface returns HTTP 408 on timeouts.
• Implementing HTTP basic auth and ensuring consistent escaping of VARCHAR fields in CSV exports.

Summary

Keep your eyes peeled for more good stuff coming from QuestDB. This year, we're taking leaps. For questions and comments, swing by our vibrant public Slack or laid back community forum. We'd love to hear from you.