Summer time! π For some, that's sunshine, beaches, and lotion, all that fun stuff. Or, perhaps, hiding inside near the air conditioner because it's hot! For QuestDB, it's another set of improvements. This release contains a bundle of smaller improvements, with new features laying down major groundwork. And - of course - the usual motion of performance improvements. Hot indeed. π₯
Upgrade! Apply liberal amounts of sunscreen!
For the full notes including PR links, checkout GitHub.
Breaking changes π₯β
-
QuestDB no longer supports FreeBSD. We test major browser versions on each release. That means everything we write needs to account for multiple OS paradigms. FreeBSD has very small market share, and more often than not we'd trip over edge cases. To speed things up, we've made the decision to drop support. This means FreeBSD is no longer tested during development.
-
NFS is now an unsupported file system. QuestDB will refuse to start if running on an NFS disk. We recommend using ZFS. For a full list of supported filesystems, checkout the Capacity Planning documentation.
New features π£β
JSON supportβ
The json_extract()
function is the first step towards robust JSON support.
Leveraging the simdjson library by
Daniel Lemire, it allows extracting fields from a
JSON document and storing them into VARCHAR columns. There is some performance
overhead, so we recommend using it to test a schema before moving from JSON to
native column types.
SELECT
json_extract(trade_details, '$.quantity')::long quantity,
json_extract(trade_details, '$.price')::double price,
json_extract(trade_details, '$.executions[0].timestamp')::timestamp first_ex_ts
FROM
trades
WHERE
json_extract(trade_details, '$.exchange') = 'NASDAQ'
Read more in the documentation.
New financial functionsβ
Introducing the mid_price
function and spread
function for financial
calculations. Simple, but effective enhancements for financial data processing.
Smarter Web Consoleβ
Vague errors aren't helpful. The Web Console now helps you self-troubleshoot common database configuration issues with clear visual indicators to identify and resolve problems.
For example, what if one of our tables is suspended?
![The UI shows a column of table names, which are clearly marked in colour that there is an issue.](/img/blog/2024-07-16/error-web-console.webp)
The UI will clearly show there's an issue with the table. There is an issue with
trips
. Knowing there's an issue is half of it. But now, what can we do?
![The UI guides through fixing.](/img/blog/2024-07-16/solution-web-console.webp)
In many cases, we can identify the specific issue and offer specific guidance to help resolve it. In this case, there is no disk space. There's a link to the docs to help, and the ability to restart once you've resolved the root issue.
Look forward to more improvements in this way as we get better at providing clear erroring and resoluton steps.
HTTP Basic Authβ
QuestDB open source now offers Basic Auth to protect its HTTP endpoints.
See the documentation for configuration guidance.
Performance πβ
As always, QuestDB is a little faster, better, and stronger than before:
-
JIT-optimized IN(numeric values) operator for faster query execution.
-
Optimized aggregate queries with the LAST function on designated timestamps.
-
Speed improvements for self-join queries on symbol columns and more efficient pattern lookups for short ASCII patterns.
-
Enhancements to
GROUP BY
operations and parallel execution, particularly benefiting queries with aggregate functions likeavg()
andsum()
.
Bug fixes πβ
And we've squashedΒ some pesky bugs:
- VARCHAR column queries via
sqlx::postgres
. - Invalid column errors in
GROUP BY
withJOIN
. - Improved error handling to prevent ugly error messages when the log directory does not exist.
- Ensuring REST interface returns HTTP 408 on timeouts.
- Implementing HTTP basic auth and ensuring consistent escaping of VARCHAR fields in CSV exports.
Summaryβ
Keep your eyes peeled for more good stuff coming from QuestDB. This year, we're taking leaps. For questions and comments, swing by our vibrant public Slack or laid back community forum. We'd love to hear from you.