Insert data
This page shows how to insert data into QuestDB using different programming languages and tools.
InfluxDB Line Protocol is the recommended primary ingestion method in QuestDB and is recommended for high-performance applications.
For transactional data inserts, use the PostgreSQL wire protocol.
For operational (ad-hoc) data ingestion, the Web Console makes
it easy to upload CSV files and insert via SQL statements. You can also perform
these same actions via the HTTP REST API. For
large CSV import (database migrations), use SQL
COPY
.
In summary, these are the different options:
- InfluxDB Line Protocol
- High performance.
- Optional automatic timestamps.
- Optional integrated authentication.
- Client libraries in various programming languages.
- PostgreSQL wire protocol
- SQL
INSERT
statements, including parameterized queries. - Use
psql
on the command line. - Interoperability with third-party tools and libraries.
- SQL
- Web Console
- CSV upload.
- SQL
INSERT
statements. - SQL
COPY
for large CSV import.
- HTTP REST API
- CSV upload.
- SQL
INSERT
statements. - Use
curl
on the command line.
Here is a summary table comparing the different ways to insert data we support:
Protocol | Record Insertion Reporting | Data Insertion Performance |
---|---|---|
InfluxDB Line Protocol | Server logs; Disconnect on error | Best |
CSV upload via HTTP REST | Configurable | Very Good |
SQL INSERT statements | Transaction-level | Good |
SQL COPY statements | Transaction-level | Suitable for one-off data migration |
#
InfluxDB Line ProtocolThe InfluxDB Line Protocol (ILP) is a text protocol over TCP on port 9009
.
It is a one-way protocol to insert data, focusing on simplicity and performance.
This interface is the preferred ingestion method as it provides the following benefits:
- High-throughput ingestion
- Robust ingestion from multiple sources into tables with dedicated systems for reducing congestion
- Configurable commit-lag for out-of-order data via server configuration settings
On the InfluxDB line protocol page, you may find additional details on the message format, ports and authentication.
The Telegraf guide helps you configure a Telegraf agent to collect and send metrics to QuestDB via ILP.
tip
The ILP client libraries provide more user-friendly ILP clients for a growing number of languages.
#
ExamplesThese examples send a few rows of input. These use client libraries as well as raw TCP socket connections, when a client library is not available.
- Python
- Go
- Java
- NodeJS
- C#
- C
- C++
- Rust
- Ruby
- PHP
Java client library docs and Maven artifact.
Maven
<dependency>
<groupId>org.questdb</groupId>
<artifactId>questdb</artifactId>
<version>6.5</version>
</dependency>
Gradle
compile group: 'org.questdb', name: 'questdb', version: '6.5'
NodeJS client library repo.
#
PostgreSQL wire protocolQuestDB also supports the same wire protocol as PostgreSQL, allowing you to connect and query the database with various third-party pre-existing client libraries and tools.
You can connect to TCP port 8812
and use both INSERT
and SELECT
SQL
queries.
PostgreSQL wire protocol is better suited for applications inserting via SQL programmatically as it provides parameterized queries, which avoid SQL injection issues.
tip
InfluxDB Line Protocol is the recommended primary
ingestion method in QuestDB. SQL INSERT
statements over the PostgreSQL offer
feedback and error reporting, but have worse overall performance.
Here are a few examples demonstrating SQL INSERT
queries:
- psql
- Python
- Java
- NodeJS
- Go
- Rust
Create the table:
Insert row:
Query back:
Note that you can also run psql
from Docker without installing the client
locally:
This example uses the pg
package which
allows for quickly building queries using Postgres wire protocol. Details on the
use of this package can be found on the
node-postgres documentation.
This example uses naive Date.now() * 1000
inserts for Timestamp types in
microsecond resolution. For accurate microsecond timestamps, the
process.hrtime.bigint()
call can be used.
This example uses the pgx driver and toolkit for PostgreSQL in Go. More details on the use of this toolkit can be found on the GitHub repository for pgx.
The following example shows how to use parameterized queries and prepared statements using the rust-postgres client.
#
Web ConsoleQuestDB ships with an embedded Web Console running
by default on port 9000
.
SQL statements can be written in the code editor and executed by clicking the Run button. Note that the web console runs a single statement at a time.
For inserting bulk data or migrating data from other databases, see large CSV import.
#
HTTP REST APIQuestDB exposes a REST API for compatibility with a wide range of libraries and
tools. The REST API is accessible on port 9000
and has the following
insert-capable entrypoints:
Entrypoint | HTTP Method | Description | API Docs |
---|---|---|---|
/imp | POST | Import CSV data | Reference |
/exec?query=.. | GET | Run SQL Query returning JSON result set | Reference |
For details such as content type, query parameters and more, refer to the REST API docs.
/imp
: Uploading Tabular Data#
tip
InfluxDB Line Protocol is the recommended primary ingestion method in QuestDB. CSV uploading offers insertion feedback and error reporting, but has worse overall performance.
See /imp
's atomicity
query
parameter to customize behavior on error.
Let's assume you want to upload the following data via the /imp
entrypoint:
- CSV
- Table
col1 | col2 | col3 |
---|---|---|
a | 10.5 | true |
b | 100 | false |
c | NULL | true |
You can do so via the command line using cURL
or programmatically via HTTP
APIs in your scripts and applications.
By default, the response is designed to be human-readable. Use the fmt=json
query argument to obtain a response in JSON. You can also specify the schema
explicitly. See the second example in Python for these features.
- cURL
- Python
- NodeJS
- Go
This example imports a CSV file with automatic schema detection.
This example overwrites an existing table and specifies a timestamp format and a designated timestamp column. For more information on the optional parameters to specify timestamp formats, partitioning and renaming tables, see the REST API documentation.
This first example shows uploading the data.csv
file with automatic schema
detection.
The second example creates a CSV buffer from Python objects and uploads them with a custom schema. Note UTF-8 encoding.
The fmt=json
parameter allows us to obtain a parsable response, rather than a
tabular response designed for human consumption.
/exec
: SQL INSERT
Query#
The /exec
entrypoint takes a SQL query and returns results as JSON.
We can use this for quick SQL inserts too, but note that there's no support for parameterized queries that are necessary to avoid SQL injection issues.
tip
Prefer the PostgreSQL interface if you are generating sql programmatically.
Prefer ILP if you need high-performance inserts.
- cURL
- Python
- NodeJS
- Go
The node-fetch
package can be installed using npm i node-fetch
.