DevStories is our brand-new series. We interview developers in our community to understand how they build applications with QuestDB. For the first post of this series, we are thrilled to interview Kevin Kamto, Co-founder at AthleteX.
My name is Kevin Kamto. I'm one of the core contributors and co-founders of AthleteX. That means that on a day to day basis, I operate as the head of sales and point of coordination for the team.
I've been working in the IoT space as a developer for many years. For a cool project I worked on, we needed to create a device that measured the WiFi usage in the company to observe the day to day patterns. That's when I got introduced to InfluxDB, one of the time series database pioneers at that time.
This is the backstory: during the middle of COVID-19, there were a lot of markets seeing red. So we asked ourselves, what is an investor to do during this time? As someone who loves sports like soccer and basketball, I thought that if I could put my money towards an athlete, I could keep it safe.
This blog post is not investment advice. 😇
The idea of AthleteX is that you can invest in the performance of your favorite athletes. Essentially, we created a fantasy sports prediction market. You can go to the platform to either long or short the performance of an athlete by trading the Athlete Performance Tokens (APTs). The price of an APT is based on an athlete's in-game statistics. Taking baseball as an example, we determine price using the Wins Above Replacement (WAR) formula, and the price is updated in real-time.
Using MLB (baseball) as an example, some in-game metrics such as bats and home runs are generated every minute or every five minutes. We must store all that information over time. Coming from an IoT background, the problem is not new to me. So instead of going with something like MySQL or MongoDB, which didn't make a lot of sense, I knew that a time series database was what we needed.
QuestDB is faster than its competitors and goes above and beyond what we need it to handle at the current stage. It's also convenient to use SQL as the query language. At the moment, we store thousands of time series every minute and both the ingestion and querying performance has been satisfying. At scale, we anticipate hundreds of millions of data points and expect QuestDB to handle it smoothly.
QuestDB is the core foundation of our entire backend infrastructure. Due to the feed-type nature of our application and the need to look up athlete stats very fast, we need a database to manage large time series data sets while providing fast query speed.
Our data providers have all the sports related data for each season, sport, and player. And we have some Python scripts to grab all the data periodically, calculate the book value based on the WAR formula, and store it in QuestDB. QuestDB is the base layer that allows us to handle this kind of throughput. At the moment, we're storing nearly 2 millions records in QuestDB.
Further down in the data pipeline, we built an API using Kotlin to query QuestDB and show the charts on our frontend. And from this point on, it's more about the DeFi protocol than QuestDB, you can read our lite paper to get a deeper understanding on how our system was designed.
We often use simple
SELECT statements together with
ORDER BY. We
also dabbled with
SAMPLE BY and the performance has been exemplary.
We wanted to allow some users to query QuestDB directly using the web console or some dashboards, without having access to drop or alter any tables. It was not possible to do that directly since it's lacking a security layer or built-in authentication. We ended up writing our own API for this purpose.
Insights from the QuestDB team: since
QuestDB 6.3, we
implemented a new configuration
pg.security.readonly to toggle the read-only
mode for PGWire. Similarily, the existing
http.security.readonly can be
applied to the web console.
High availability is one thing we look forward to seeing in QuestDB. For now we try to scale the resources with the single instance but it come at the expense of some downtime. Replication is an important feature to ensure that the data is echoed correctly across all the replicas.
We could also benefit a lot if QuestDB can work smoothly with Metabase or something similar to that. We've tried to create dashboards with Grafana but it doesn't appear to be as human friendly as products like Metabase, especially for the non-technical end users that we have.
AtheleteX 1.0.0 has just been released on Sept 10, 2022. If you're interested in learning more about the project, check out our release announcement!