Invezo logoCase study

Invezo

QuestDB is the backbone behind analytical capabilities at Invezo, which aims to become Bloomberg for digital assets.

QuestDB is the backbone behind analytical capabilities at Invezo, which aims to become Bloomberg for digital assets.

Briefcase iconUse case: Market data

Globe iconIndustry: Fintech

Flag iconDeployment: QuestDB Cloud

The team

Invezo is a fast growing startup that aims to become the gold standard for digital assets analytics. Traditional companies in equity markets are assessed and valued based on established frameworks; their coverage from equity research companies is plentiful and all of this information is aggregated on platforms such as the Bloomberg Terminal. It's a different situation for crypto assets, which is a brand new asset class. There is a lack of tools to analyze market price behavior and underlying valuations. Invezo pulls on-chain and off-chain metrics to understand the behavior of crypto assets to a much deeper level, while offering superior charting capabilities and also providing an API.

Emmett Miller, co-founder and CEO, Invezo, tells us about his journey building his company using QuestDB Cloud.

The need for a time-series database to deliver real-time analytics to end users

All of the data in the Invezo platform is time series. We track all of our metrics over time. There are plenty of metrics to track: cryptocurrency token price, GitHub commits, reddit subscribers, on-chain wallet transactions. This data is then exposed to the end user of the platform in real-time. The latency needs to be as low as possible to guarantee a good user experience. We want the dashboards to refresh live in milliseconds rather than seconds. This may seem a trivial thing to achieve but it is not: users zooming in and out of our dynamic charts are performing heavy operations for the underlying database to compute. New data also flows in real-time, requiring aggregations on the fly. Performant time-series databases are crucial to cope with the ingestion rate: we currently pull market data for hundreds of cryptocurrencies from more than nine different exchanges. We also need very fast querying capabilities to power our live dashboards and analytics.

Example of live dashboards and analytics in real time. Each refresh is a SQL query for QuestDB.

Example of live dashboards and analytics in real time. Each refresh is a SQL query for QuestDB.

QuestDB deployment within the Invezo stack

We collect data from crypto exchanges APIs and build our main application in Python. We use the official QuestDB Python client library to send the data via the InfluxDB Line Protocol ("ILP") over TCP. Each query is computed programmatically via PGWire using SQL and feeds real-time data, aggregations and downsampled data to plot out charts.

QuestDB has built time-series specific syntax to make queries less verbose. For example, SAMPLE BY automatically downsamples the data for a given interval. An example of SQL query to compute the number of twitter mentions over time is the following:

SELECT ts, sum(value)
FROM 'twitter_mentions'
WHERE symbol='btc' AND ts >= dateadd('d', 14, now)
SAMPLE BY 10m
Example of live dashboards and analytics in real time. Each refresh is a SQL query for QuestDB.

Example of live dashboards and analytics in real time. Each refresh is a SQL query for QuestDB.

We started with AWS RD but the database was too slow for our use case as queries would take up to 6 minutes to come back. We then looked into MongoDB, which recently released time-series capabilities. Here again, slow ingestion and queries made our application unusable (see a comparison with QuestDB toward the end of the case study). In addition, we had a crack at InfluxDB but their query language "Flux" was a show stopper because it is unintuitive.

Here is a comparison of the same query between MongoDB for time series and QuestDB:

MongoDB Query
def basicAggregation():
pipeline = [
{"$match": {"metadata.pair": "btc-usdt", "metadata.exchange": "binance"}},
{"$sort": {"time": -1}},
{"$limit": 50000}
]
results = col.aggregate(pipeline, hint="Main Index", allowDiskUse=True)
values = []
for i in results:
values.append(i)
return values
start = time.time()
basicAggregation()
print(time.time() - start)

MongoDB query execution time: 5.4 seconds

QuestDB Query
SELECT ts, first(open), max(high), min(low), last(close)
FROM 'ohlc_all'
WHERE pair='btc-usd'
SAMPLE BY 5m
LIMIT 50000

QuestDB query execution time: 0.1 seconds

We saw the light at the end of the tunnel when we found out about QuestDB. We liked it for its superior ingestion rates and the ability to query data on the fly. On top of the superior performance we are impressed by how vibrant the community is; we asked a few questions on the QuestDB Slack and got a response immediately. We love SQL and wanted to build the queries for our application with a familiar language.

SAMPLE BY queries, which are on the fly

SAMPLE BY queries, which are on the fly

QuestDB Cloud: why we chose the fully hosted solution

We initially deployed QuestDB open source via the AWS marketplace. With QuestDB open source, we do not have the luxury of web console authentication, although we knew that a workaround with setting NGINX as a proxy was possible. We also did not have the bandwidth nor internal resources to properly monitor and manage the instances. It made sense to us that the instances should be overseen by the QuestDB creators, who know the product inside out.

By offloading the management and monitoring responsibilities to the QuestDB team, we gained precious hours to focus on our business. We also benefit from authentication for the web console, and scheduled snapshots without any downtime at the click of a button. The UI is intuitive and the team is shipping new features every week. I am excited about the upcoming cold storage integration with AWS S3, which should free a lot of space on our EBS volume.

Our customers value a low-latency API, so speed is extremely important to us. With QuestDB, our ingestion rate is 5x faster and query execution time went from minutes to milliseconds

Emmett Miller, Co-founder, Invezo