InfluxDB Line Protocol Overview

QuestDB implements the InfluxDB Line Protocol to ingest data.

The InfluxDB Line Protocol is for data ingestion only.

For building queries, see the Query & SQL Overview.

Each ILP client library also has its own language-specific documentation set.

This supporting document thus provides an overview to aid in client selection and initial configuration:

  1. Client libraries
  2. Configuration
  3. Authentication
  4. Transactionality caveat

Client libraries#

The quickest way to get started is to select your library of choice.

From there, its documentation will carry you through to implementation.

Client libraries are available for several languages:

If you'd like more context on ILP overall, please continue reading.

Enable or disable ILP#

If going over HTTP, ILP will use shared HTTP port 9000 (default) if the following is set in server.conf:

line.http.enabled=true

Configuration#

The HTTP receiver configuration can be completely customized using QuestDB configuration keys for ILP.

Configure the thread pools, buffer and queue sizes, receiver IP address and port, load balancing, and more.

For more guidance in how to tune QuestDB, see capacity planning.

Authentication#

note

Using QuestDB Enterprise?

Skip to advanced security features instead, which provides holistic security out-of-the-box.

InfluxDB Line Protocol supports authentication.

A similar pattern is used across all client libraries.

This document will break down and demonstrate the configuration keys and core configuration options.

Once a client has been selected and configured, resume from your language client documentation.

Configuration strings#

Configuration strings combine a set of key/value pairs.

Assembling a string connects an ILP client to a QuestDB ILP server.

The standard configuration string pattern is:

schema::key1=value1;key2=value2;key3=value3;

It is made up of the following parts:

  • Schema: One of the specified schemas in the base values section below
  • Key=Value: Each key-value pair sets a specific parameter for the client
  • Terminating semicolon: A semicolon must follow the last key-value pair

Client parameters#

Below is a list of common parameters that ILP clients will accept.

These params facilitate connection to QuestDB's ILP server and define client-specific behaviors.

Some are shared across all clients, while some are client specific.

See the Usage section for write examples that use these schemas.

Base parameters#

  • schema: Specifies the transport method, with support for: http, https, tcp & tcps
  • addr: The address and port of the QuestDB server.
  • username: Username for authentication (HTTP Basic or TCP).

Sensitive parameters#

Warning! Please remember to treat the following as sensitive.

Exposing these tokens could expose your database.

  • password: Password for HTTP Basic authentication.
  • token, token_x, token_y: Authentication tokens and their parts for TCP authentication.
    • token_x & token_y only needed for C/C++/Rust & Python clients.

Auto-flushing behavior#

  • auto_flush: Enable or disable automatic flushing (on/off).

    • Default is “on” for clients that support auto-flushing (All except C, C++ & Rust).
  • auto_flush_rows: Auto-flushing is triggered above this row count.

    • Defaults to 75,000 if unspecified
    • If set, this implies “auto_flush=on”.
  • auto_flush_bytes Auto-flushing is triggered above this buffer size.

    • Python only.
    • Errors if “auto_flush_rows” is set.
    • Disabled by default.

Network configuration#

Optional.

  • bind_interface: Specify the local network interface for outbound connections.
    • Not to be confused with the QuestDB port in the addr param.

TLS configuration#

  • tls_verify: Toggle verification of TLS certificates. Default is on.
  • tls_roots: Specify the source of bundled TLS certificates.
    • The defaults and possible param values are client-specific.
      • In Rust and Python this might be “webpki”, “os-certs” or a path to a “pem” file.
      • In Java this might be a path to a “jks” trust store.
      • tls_roots_password Password to a configured tls_roots if any.
        • Passwords are sensitive! Manage appropriately.
  • tls_ca: Path to single certificate authourity, not supported on all clients.
    • Java for instance would apply tls_roots=/path/to/Java/key/store

Transactionality caveat#

As of writing, the HTTP endpoint does not provide full transactionality in all cases.

Specifically:

  • If an HTTP request contains data for two tables and the final commit fails for the second table, the data for the first table will still be committed. This is a deviation from full transactionality, where a failure in any part of the transaction would result in the entire transaction being rolled back.

  • When adding new columns to a table, an implicit commit occurs each time a new column is added. If the request is aborted or has parse errors, this commit cannot be rolled back.


Something missing? Page not helpful? Please suggest an edit on GitHub.