Amazon Timestream

Let's connect a nodejs app in one Amazon VPC with a Amazon Timestream managed InfluxDB database in another Amazon VPC. We’ll create an end-to-end encrypted Ockam Portal to InfluxDB.

To understand the details of how end-to-end trust is established, and how the portal works even though the two networks are isolated with no exposed ports, please read: “How does Ockam work?”

Run

This example requires Bash, Git, AWS CLI. Please set up these tools for your operating system. In particular you need to login to your AWS account.

Amazon Timestream for InfluxDB was added very recently. To run this example, please install the latest version of AWS CLI.

Then run the following commands:

# Clone the Ockam repo from Github.
git clone --depth 1 https://github.com/build-trust/ockam && cd ockam

# Navigate to this example’s directory.
cd examples/command/portals/databases/influxdb/amazon_timestream/aws_cli

# Run the example, use Ctrl-C to exit at any point.
./run.sh

If everything runs as expected, you'll see the message: The example run was successful 🥳

Walkthrough

The run.sh script, that you ran above, and its accompanying files are full of comments and meant to be read. The example setup is only a few simple steps, so please take some time to read and explore.

Administrator

• The run.sh script calls the run function which invokes the enroll command to create an new identity, sign into Ockam Orchestrator, set up a new Ockam project, make you the administrator of this project, and get a project membership credential.

• The run function then generates two new enrollment tickets. The tickets are valid for 10 minutes. Each ticket can be redeemed only once and assigns attributes to its redeemer. The first ticket is meant for the Ockam node that will run in Metrics Corp.’s network. The second ticket is meant for the Ockam node that will run in Datastream Corp.’s network.

• In a typical production setup an administrator or provisioning pipeline generates enrollment tickets and gives them to nodes that are being provisioned. In our example, the run function is acting on your behalf as the administrator of the Ockam project.

• The run function passes the enrollment tickets as variables of the run scripts provisioning Metrics Corp.'s network and Datastream Corp.'s network.

Metrics Corp

First, the metrics_corp/run.sh script creates a network to host the database:

• It creates a VPC and tags it.

• It creates an Internet gateway and attaches it to the VPC.

• It creates a route table and a route to the Internet via the gateway.

• It creates a subnet and associates it with the route table.

• It creates a security group which allows:

  • TCP egress to the Internet.

  • Ingress to InfluxDB from within the subnet.

  • SSH ingress to provision EC2 instances.

Then, the metrics_corp/run.sh script creates a InfluxDB database using Timestream. Next the script creates an EC2 instance. This instance runs an Ockam TCP Outlet.

• It selects an AMI.

• It then starts an instance using this AMI and a start script based on run_ockam.sh where:

  • ENROLLMENT_TICKET is replaced by the enrollment ticket created by the administrator and given as a parameter to run.sh.

  • INFLUXDB_ADDRESS is replaced by the database address that we previously saved.

• It tags the created instance and waits for it to be available.

When EC2 starts the instance, it executes the run_ockam.sh script:

• It installs the Influxdb client and configures it.

• It generates an InfluxDB auth token to send to Datastream Corp and saves it to file.

• It installs the ockam command.

• It uses the enrollment ticket to create a default identity and make it a project member.

• It then creates an Ockam node with:

  • A TCP outlet.

  • An access control policy associated to the outlet. The policy authorizes only identities with a credential attesting to the attribute influxdb-inlet="true".

  • A a relay that can forward TCP traffic to the TCP outlet.

Datastream Corp

First, the datastream_corp/run.sh script creates a network to host the nodejs application:

• It creates a VPC and tags it.

• It creates an Internet gateway and attaches it to the VPC.

• It creates a route table and a route to the Internet via the gateway.

• It creates a subnet and associates it with the route table.

• It creates a security group that allows:

  • TCP egress to the Internet,

  • SSH ingress to provision EC2 instances.

Next, the script creates an EC2 instance. This instance runs an Ockam TCP Inlet.

• It selects an AMI.

• It then starts an instance using that AMI and a start script based on run_ockam.sh in which the:

  • The variable ENROLLMENT_TICKET is replaced by the enrollment ticket created by the administrator and given as a parameter to run.sh.

When EC2 starts the instance, it executes the run_ockam.sh script:

• It installs ockam command.

• It uses the enrollment ticket is used to create a default identity and make it a project member.

• It then creates an Ockam node with:

  • A TCP inlet.

  • An access control policy associated with the inlet. The policy authorizes identities with a credential attesting to the attribute influxdb-outlet="true".

Next datastream_corp/run.sh waits for the instance to be ready and provisions it using SSH:

• It copies app.js and token.txt into the instance using SCP

• It then runs a script, using SSH, which:

  • Installs nodejs.

  • Installs the InfluxDB client library.

  • Starts the nodejs application.

Finally, the nodejs application is started:

• It connects to the Ockam inlet at localhost:8086.

• It inserts a few system metrics into a bucket and retrieves them back to show that the connection with the InfluxDB database is working.

Recap

We connected a nodejs app in one virtual private network with a InfluxDB database in another virtual private network over an end-to-end encrypted portal.

Sensitive business data in the InfluxDB database is only accessible to Metrics Corp. and Datastream Corp. All data is encrypted with strong forward secrecy as it moves through the Internet. The communication channel is mutually authenticated and authorized. Keys and credentials are automatically rotated. Access to connect with InfluxDB can be easily revoked.

Datastream Corp. does not get unfettered access to Metrics Corp.’s network. It gets access only to query InfluxDB. Metrics Corp. does not get unfettered access to Datastream Corp.’s network. It gets access only to respond to queries over a tcp connection. Metrics Corp. cannot initiate connections.

All access controls are secure-by-default. Only project members, with valid credentials, can connect with each other. NAT’s are traversed using a relay and outgoing tcp connections. Metrics Corp. or Datastream Corp. don’t expose any listening endpoints on the Internet. Their networks are completely closed and protected from any attacks from the Internet.

Cleanup

To delete all AWS resources:

./run.sh cleanup