This guide describes how to create an AWS Amazon Machine Image (AMI) based on Amazon Linux with QuestDB installed and uses the official QuestDB packer AMI with a template that can be used as a point of reference to create your own AMIs.
This document also covers details on applying networking rules via security groups to allow access to the REST API and web console publicly accessible or by whitelisted IPs, and how to enable logging to CloudWatch via the AWS CLI.
- An Amazon Web Services account
- AWS CLI for programmatic access to AWS resources
- Packer for building and provisioning AMIs
To check that the AWS CLI is configured correctly, run the following command:
The configuration should be returned showing which profile, credentials and region is configured:
If no configuration has been set up for the AWS CLI yet, run the following command to set up the CLI:
To configure and build the machine image, clone the official GitHub repository with a Packer template:
A placeholder configuration file for QuestDB server settings can be found at the following location:
The included configuration does not override any settings, therefore default server configuration will be used in the AMI. For a comprehensive list of properties which may be set, refer to the QuestDB server configuration documentation.
src directory contains a
template.json file which may be used as a
starting point for your own AMI.
The template uses Amazon Linux 2 with the
t3.micro instance type. The default
template variables may be passed on the command line using
-var <var_name>=<var_value> to specify a specific region and AMI base name,
The following command will build the QuestDB machine image and create it in the
eu-central-1 region with the name
Log output from Packer will show a Packer Builder EC2 instance creating the image:
To view the details of the image, the following AWS CLI command will describe AMIs created by the current AWS account:
The QuestDB image should be listed as one of the available images. Make a note
ImageId value which will be referred to in the following section as
Instances using this AMI with QuestDB installed can be directly launched from
the CLI. For convenience, we will first allow networking on instance creation
via a security group. For this guide, we will enable port
9000 which allows
Make a note of the security group ID returned from this command and pass it as
<sec_group_id> variable below to allow ingress on TCP port
9000 for IPV4
In this example, ingress on port
9000is open for requests originating from any IP. This is a security group configuration used for illustrative purposes only. When deploying QuestDB to production, only trusted or required public IP addresses should be allowed.
Users may also want to enable networking for other types of traffic. QuestDB listens for PostgreSQL wire protocol by default on port
8812and InfluxDB Line Protocol on ports
9009for TCP and UDP.
Launch the AMI with the security group attached:
To find the public IP address of the QuestDB instance, run the following
command, replacing the instance ID in the
filters parameter (
The CLI will return the public IP of EC2 instances running the QuestDB AMI created in the preceding section. To verify that the web console of this running instance is active:
- Copy the External IP of the instance
- Navigate to
<external_ip>:9000in a browser
Alternatively, a request may be sent against the REST API exposed on port
For more information on using this functionality, see the official documentation for using the QuestDB REST API.
The AMI uses Linux logrotate utility to
automatically trim and archive logs generated by QuestDB. The
AWS CloudWatch agent
is also pre-installed and configured to start sending log messages. To make the
logs available on your CloudWatch dashboard, an instance profile with the
must be associated with the EC2 instance running QuestDB.
The following two JSON documents may be copied directly without modifications to
assign an IAM role with the correct permissions for logging. Create a file
trust_policy.json file with the following contents which allows EC2 instances
to assume the role:
Create a permission policy document
permission_policy.json which provides
permissions to write to CloudWatch:
Create the IAM role using these documents:
Create an instance policy, attach the IAM role, and associate the IAM role with the EC2 instance running QuestDB:
Create a CloudWatch log group and log stream:
To read the latest log events from this stream from the CLI, use the following command:
A JSON object containing the most recent events will be returned: