🔗 Ockam's docs are LLM-ready: You can use https://docs.ockam.io/llms-full.txt to prompt large language models to understand and reason about Ockam using official documentation.

How to Use Ockam's llms-full.txt with ChatGPT and Cursor

You can prompt AI tools to use Ockam's documentation by referencing our LLM-ready index. This helps large language models answer your questions using trusted, up-to-date information.

For ChatGPT (Pro users with GPT-4o)

1. Open and select GPT-4o.

2. Paste the following prompt:

3. Then ask your question. For example:

ChatGPT will now reference the docs listed in llms-full.txt to give more accurate answers.

✅ For Cursor (AI coding editor)

Cursor supports web context and documentation lookups.

1. Open Cursor and activate the chat (Cmd+K or click the Chat icon).

2. Type:

3. Ask follow-up questions like:

Cursor will incorporate the docs into its responses, making code completions and suggestions more relevant to Ockam.

🔍 What is llms-full.txt?

This file is part of the Model Context Protocol (MCP) — a standard that allows developers to expose their full documentation to AI tools in a structured way.

Ockam is a popular that empowers you to build secure-by-design apps that can trust data-in-motion. Hundreds of developers have contributed to building, reviewing the codebase over the past 5 years.

With Ockam:

• Impossible connections become possible. Establish secure channels between systems in private networks that previously could not be connected because it is either too difficult or insecure.

• All public endpoints become private. Connect your applications and databases without exposing anything publicly.

At its core, Ockam is a toolkit for developers to build applications that can create end-to-end encrypted, mutually authenticated, secure communication channels:

Ockam Command is our command line interface to build secure by design applications that can trust all data in motion. It makes it easy to orchestrate end-to-end encryption, mutual authentication, key management, credential management, and authorization policy enforcement – at a massive scale.

No more having to design error-prone ad-hoc ways to distribute sensitive credentials and roots of trust. Ockam's integrated approach takes away this complexity and gives you simple tools for:

End-to-end data authenticity, integrity, and privacy in any communication topology

Nodes and Workers

Please select specific marketplace listings to view

Ockam has been audited and verified by experts

A team of security and cryptography experts, from Trail of Bits, conducted an extensive review of Ockam’s protocols. Trail of Bits is renowned for their comprehensive third-party audits of the security of many other critical projects, including Kubernetes and the Linux kernel.

The auditors from Trail of Bits conducted in-depth, manual analysis, and formal modeling of the security properties of Ockam’s protocols. After this review was complete, they highlighted:

Ockam’s protocols use robust cryptographic primitives according to industry best practices. None of the identified issues pose an immediate risk to the confidentiality and integrity of data handled by the system in the context of the two in-scope use cases. The majority of identified issues relate to information that should be added to the design documentation, such as threat model details and increased specification for certain aspects.

— Trail of Bits

Download the unabridged audit by Trail of Bits:

In this video we cover:

• Authentication and Authorization: What's the difference?

• Introduction to Ockam's Authentication and Authorization protocols

• The developer experience of Ockam is 3 commands

• How to create secure connections to customer data

• Routing protocols in Ockam

• Secure Channels in Ockam

• Attribute-Based Access Control (ABAC) in Ockam

• Revocation and rotation of credentials in Ockam

• Identifiers, Identity, Keys, and Credentials are foundational roots of Trust in Ockam

• Cryptography of change events in Ockam

• Scaling Trust to Enterprise scale

In order to make decisions about trust, we must authenticate senders of messages.

Vaults

Ockam Identities authenticate by cryptographically proving possession of specific secret keys. Ockam Vaults safely store these secret keys in cryptographic hardware and cloud key management systems.

You can create a vault as follows:

This command will, by default, create a file system based vault, where your secret keys are stored at a specific file path.

Vaults are designed to be used in a way that secret keys never have to leave a vault. There is a growing base of Ockam Vault implementations in the that safely store secret keys in specific KMSs, HSMs, Secure Enclaves etc.

Identities

Ockam Identities are unique, cryptographically verifiable digital identities.

You can create new identities, by typing:

The secret keys belonging to this identity are stored in the specified vault. This can be any type of vault - File Vault, AWS KMS, Azure KeyVault, YubiKey etc. If no vault is specified, the default vault is used. If a default vault doesn't exist yet, a new file systems based vault is created, set as default, and then used to generate secret keys.

To ensure privacy and eliminate the possibility of correlation of behavior across trust contexts, we've made it easy to generate and use different identities and identifiers for separate trust contexts.

Secret Keys

Each Ockam Identity starts its life by generating a secret key and its corresponding public key. Secret keys, must remain secret, while public keys can be shared with the world.

Ockam Identities support two types of Elliptic Curve secret keys that live in vaults - Curve25519 or NIST P-256.

Identifiers

Each Ockam Identity has a unique public identifier, called the Ockam Identifier of this identity:

This Identifier is generated by hashing the first public key of the Identity.

Change History

Ockam Identities can periodically rotate their keys to indicate that the latest public key is the one that should be used for authentication. Each Ockam Identity maintains a self-signed change history of key rotation events, you can see this full history by running:

Identifier Authentication

Authentication, within Ockam, starts by proving control of a specific Ockam Identifier. To prove control of a specific Identifier, the prover must present the identifier, the full signed change history of the identifier, and a signature on a challenge using the secret key corresponding to the latest public key in the identifier's change history.

Next, let's combine everything we've learnt so far to create mutually authenticated and end-to-end encrypted that guarantee data authenticity, integrity, and confidentiality.

Ockam is composed of a collection of cryptographic and messaging protocols. These protocols make it possible to create private and secure by design applications that provide end-to-end application layer trust it data. The following pages explain, in detail, how each of the protocols work:

• Nodes and Workers

• Routing and Transports

Ockam has been audited and verified by experts

A team of security and cryptography experts, from Trail of Bits, conducted an extensive review of Ockam’s protocols. Trail of Bits is renowned for their comprehensive third-party audits of the security of many other critical projects, including Kubernetes and the Linux kernel.

The auditors from Trail of Bits conducted in-depth, manual analysis, and formal modeling of the security properties of Ockam’s protocols. After this review was complete, they highlighted:

Ockam’s protocols use robust cryptographic primitives according to industry best practices. None of the identified issues pose an immediate risk to the confidentiality and integrity of data handled by the system in the context of the two in-scope use cases. The majority of identified issues relate to information that should be added to the design documentation, such as threat model details and increased specification for certain aspects.

— Trail of Bits

Download the unabridged audit by Trail of Bits:

Ockam Rust crates are a library of tools to build secure by design applications for any environment – from highly scalable cloud infrastructure to tiny battery operated microcontroller based devices. They make it easy to orchestrate end-to-end encryption, mutual authentication, key management, credential management, and authorization policy enforcement – at massive scale.

No more having to think about creating unique cryptographic keys and issuing credentials to your fleet of application entities. No more designing ways to safely store secrets in hardware and securely distribute roots of trust.

End-to-end data authenticity, integrity, and privacy in any communication topology

• Create end-to-end encrypted, authenticated secure channels over any transport topology.

• Create secure channels over multi-hop, multi-protocol routes over TCP, UDP, WebSockets, BLE, etc.

• Provision encrypted relays for applications distributed across many edge, cloud and data-center private networks.

• Make any protocol secure by tunneling it through mutually authenticated and encrypted portals.

Identity-based, policy driven, application layer trust – granular authentication and authorization

• Generate cryptographically provable unique identities.

• Store private keys in safe vaults - hardware secure enclaves and cloud key management systems.

• Operate scalable credential authorities to issue lightweight, short-lived, revokable, attribute-based credentials.

• Onboard fleets of self-sovereign application identities using secure enrollment protocols.

A step by step introduction

Ockam Rust crates provide the above collection of composable building blocks. In a step-by-step hands-on guide let’s walk through each building block to understand how you can use them to build end-to-end trustful communication for any application in any communication topology.

The first step is to install Rust and create a cargo project called hello_ockam We’ll use this project to try out various examples.

I've been fortunate to be part of some amazing teams that have had even larger communities around the products they're building. That kind of success rarely happens by accident and a great product alone is not enough to make it happen. It requires a lot of intentional nurturing of those earliest of adopters, lots of listening to people, supporting them, making yourselves and the project approachable and accessible. Those early years can be really hard but the payoff is so exciting when you look around and realize millions of people are using the products you've been building. Getting to be part of that growth story again is one of the reasons I joined Ockam! So I thought it was a good excuse to unpack some of the ways the team have been able to build the success they've had so far.

Be a small part of an existing excited community

Back in 2005/2006 I was fortunate enough to find myself exploring ruby as a language. Whatever your thoughts of the language itself, the community around it back then was incredible. So welcoming. So supportive. They even had an acronym of MINASWAN that they'd reference in forums, it stood for "Matz is nice, so we are nice". Matz being the creator of the language and so his soft demeanor was used as something to role model and take the heat out of potential flame wars. Then Rails arrived on the scene and brought with it a whole new level of excitement. It's opinionated approach to web development showed a whole new level of productivity was possible. Then Heroku arrived and did the same for deploying and running those apps at scale. The language, the tools, the community. It was like each layered on top of each other, each amplifying the excitement and impact of the previous. It was intoxicating to be part of.

In the previous section, we learned how Ockam Routing and Transports create a foundation for end-to-end application layer protocols. When discussing Transports, we put together a specific example communication topology – a transport bridge.

Bridges

Node n1 wishes to access a service on node n3, but it can't directly connect to n3. This can happen for many reasons, maybe because n3 is in a separate IP subnet, or it could be that the communication from n1 to n2 uses UDP while from n2 to n3 uses TCP or other similar constraints. The topology makes n2 a bridge or gateway between these two separate networks to enable end-to-end protocols between n1 and n3 even though they are not directly connected.

Relays

It is common, however, to encounter communication topologies where the machine that provides a service is unwilling or is not allowed to open a listening port or expose a bridge node to other networks. This is a common security best practice in enterprise environments, home networks, OT networks, and VPCs across clouds. Application developers may not have control over these choices from the infrastructure / operations layer. This is where are useful.

Relays make it possible to establish end-to-end protocols with services operating in a remote private network, without requiring a remote service to expose listening ports to an outside hostile network like the Internet.

and then try this new example:

In this example, the direction of the second TCP connection is reversed in comparison to our first example that used a bridge. n2 is the only node that has to listen for TCP connections.

Node n2 is running a relay service. n3 makes an outgoing TCP connection to n2 and requests a forwarding address from the relay service. n3 then becomes reachable via n2 at the address /service/forward_to_n3.

Node n1 connects with n2 and routes messages to n3 via its forwarding relay.

The message in the above example took the following route. This is very similar to our except for the direction of the second TCP connection. The relay worker remembers the route to back to n3. n1 just has to get the message to the forwarding relay and everything just works.

Using this simple topology rearrangement, Ockam makes it possible to establish end-to-end protocols between applications that are running in completely private networks.

We can traverse NATs and pierce through network boundaries. And since this is all built using a very simple protocol, we can have any number of transport connection hops in any transport protocol, and we can mix-match bridges with relays to create end-to-end protocols in any communication topology.

Portals

Portals make existing protocols work over Ockam Routing without changing any code in the existing applications.

Continuing from our example, create a Python-based web server to represent a sample web service. This web service is listening on 127.0.0.1:9000.

Then create a TCP Portal Outlet that makes 127.0.0.1:9000 available on worker address /service/outlet on n3. We already have a forwarding relay for n3 on n2 at service/forward_to_n3.

We then create a TCP Portal Inlet on n1 that will listen for TCP connections to 127.0.0.1:6000. For every new connection, the inlet creates a portal following the --to route all the way to the outlet. As it receives TCP data, it chunks and wraps them into Ockam Routing messages and sends them along the supplied route. The outlet receives Ockam Routing messages, unwraps them to extract TCP data and sends that data along to the target web service on 127.0.0.1:9000. It all just seamlessly works.

The HTTP requests from curl enter the inlet on n1, travel to n2, and are relayed back to n3 via its forwarding relay to reach the outlet and onward to the Python-based web service. Responses take the same return route back to curl.

The TCP Inlet/Outlet work for a large number of TCP-based protocols like HTTP. It is also simple to implement portals for other transport protocols. There is a growing base of Ockam Portal Add-Ons in our .

Recap

Ockam and combined with the ability to model and make it possible to create end-to-end, application layer protocols in any communication topology - across networks, clouds, and boundaries.

take this powerful capability a huge step forward by making it possible to apply these end-to-end protocols and their guarantees to existing applications, without changing any code!

This lays the foundation to make both new and existing applications - end-to-end encrypted and secure-by-design.

Next, let's learn how to create cryptographic and store secret keys in safe .

Data, within modern applications, routinely flows over complex, multi-hop, multi-protocol routes before reaching its end destination. It's common for application layer requests and data to move across network boundaries, beyond data centers, via shared or public networks, through queues and caches, from gateways and brokers to reach remote services and other distributed parts of an application.

Ockam is designed to enable end-to-end application layer guarantees in any communication topology.

For example, Ockam Secure Channels provide end-to-end guarantees of data authenticity, integrity, and privacy in any of the above communication topologies. In contrast, traditional secure communication implementations are typically tightly coupled with transport protocols in a way that all their security is limited to the length and duration of one underlying transport connection.

For example, most TLS implementations are tightly coupled with the underlying TCP connection. If your application's data and requests travel over two TCP connection hops TCP -> TCP, then all TLS guarantees break at the bridge between the two networks. This bridge, gateway or load balancer then becomes a point of weakness for application data.

To make matters worse, if you don't set up another mutually authenticated TLS connection on the second hop between the gateway and your destination server, then the entire second hop network – which may have thousands of applications and machines within it – becomes an attack vector to your application and its data. If any of these neighboring applications or machines are compromised, then your application and its data can also be easily compromised.

Traditional secure communication protocols are also unable to protect your application's data if it travels over multiple different transport protocols. They can't guarantee data authenticity or data integrity if your application's communication path is UDP -> TCP or BLE -> TCP.

Ockam is a simple and lightweight message-based protocol that makes it possible to bidirectionally exchange messages over a large variety of communication topologies: TCP -> TCP or TCP -> TCP -> TCP or BLE -> UDP -> TCP or BLE -> TCP -> TCP or TCP -> Kafka -> TCP or any other topology you can imagine.

Ockam adapt Ockam Routing to various transport protocols. By layering Ockam and other protocols over Ockam Routing, we can provide end-to-end guarantees over arbitrary transport topologies that span many networks and clouds.

Routing

Let's start by creating a and sending a message to a on that node.

We get a reply back and the message flow looked like this.

To achieve this, Ockam Routing Protocol messages carry with them two metadata fields: onward_route and return_route. A route is an ordered list of addresses describing a message's path travel. All of this information is carried in a really compact binary format.

Pay very close attention to the Sender, Hop, and Replier rules in the sequence diagrams below. Note how onward_route and return_route are handled as the message travels.

The above was just one message hop. We can extend this to two hops:

This very simple protocol can extend to any number of hops, try following command:

So far, we've routed messages between Workers on one Node. Next, let's see how we can route messages across nodes and machines using Ockam Routing adapters called Transports.

Transports

Ockam Transports adapt Ockam for specific transport protocol, like TCP, UDP, WebSockets, Bluetooth etc. There is a growing base of Ockam Transport implementations in the .

Let's start by exploring TCP transport. Create two new nodes: n2 and n3 and explicitly specify that they should listen on the local TCP addresses 127.0.0.1:7000 and 127.0.0.1:8000 respectively:

Next, let's create two TCP connections, one from n1 to n2 and the other from n2 to n3. Let's also add a hop for routing purposes:

Note, from the output, that the TCP connection from n1 to n2 on n1 has worker address ac40f7edbf7aca346b5d44acf82d43ba and the TCP connection from n2 to n3 on n2 has the worker address 7d2f9587d725311311668075598e291e. We can combine this information to send a message over two TCP hops.

The message in the above command took the following route:

In this example, we ran a simple uppercase request and response protocol between n1 and n3, two nodes that weren't directly connected to each other. This simple combination of Ockam Routing and Transports the foundation of end-to-end protocols in Ockam.

We can have any number of TCP hops along the route to the uppercase service. We can also easily have some hops that use a completely different transport protocol, like UDP or Bluetooth. Transport protocols are pluggable, and there is a growing base of Ockam Transport Add-Ons in our .

Recap

Ockam is a simple and lightweight message-based protocol that makes it possible to bidirectionally exchange messages over a large variety of communication topologies: TCP -> TCP or TCP -> TCP -> TCP or BLE -> UDP -> TCP or BLE -> TCP -> TCP or TCP -> Kafka -> TCP or any other topology you can imagine. Ockam adapt Ockam Routing to various transport protocols.

Together they give us a simple yet extremely flexible foundation to describe end-to-end, application layer protocols that can operate in any communication topology.

Next, let's explore how Ockam make it simple to connect existing applications across networks.

In the early days of Ockam we were developing a C library. This is the story of why, many months in, we decided to abandon tens of thousands of lines of C and rewrite in Rust.

Before we begin, I was in a recorded webinar this week together with Paul Dix, the CTO of InfluxData, where we both discussed InfluxDB’s and Ockam’s rewrites in Rust. Why the two open source projects chose to re-write, why we chose Rust as our new language, lessons we learnt along the way and more. Do checkout the recording. It was an insightful discussion.

Ockam enables developers to build applications that can trust data-in-motion. We give you simple tools to add end-to-end encrypted and mutually authenticated communication to - any application running in any environment. Your apps get end-to-end guarantees of data integrity, authenticity, and confidentiality … across private networks, between multiple clouds, through message streams in kafka – over any multi-hop, multi-protocol topology. All communication becomes end-to-end authenticated and private.

We also make the hard parts super easy to scale - bootstrap trust relationships, safely manage keys, rotate/revoke short-lived credentials, enforce attribute-based authorization policies etc. The end result is - you can build apps that have granular control over every trust and access decision - apps that are private and secure-by-design.

In 2019, we started building all of this in C. We wanted Ockam to run everywhere - from constrained edge devices to powerful cloud servers. We also wanted Ockam to be usable in any type of application - regardless of the language that application is built in.

This made C an obvious candidate. It can be compiled for 99% of computers and pretty much run everywhere (once you figure out how to deal with all the target specific toolchains). And all other popular languages can call C libraries through some form of a native function interface - so we could later provide language idiomatic wrappers for every other language: Typescript, Python, Elixir, Java etc.

The idea was we’ll keep the core of our communication centric protocols decoupled from any hardware specific behavior and have pluggable adapters for hardware we want to support. For example, there would be adapters to store secret keys in various HSMs, adaptors for various transport protocols etc.

Our plan was to implement our core as a C library. We would then wrap this C library with wrappers for other languages and run everywhere with help of pluggable hardware adapters.

Simple and Safe Interfaces

But, we also care deeply about simplicity - it's in our name. We want Ockam to be simple to use, simple to build, simple to maintain.

At Ockam’s core is a layered stack of cryptographic and message based protocols like Ockam Secure Channels and Ockam Routing. These are asynchronous, multi-step, stateful communication protocols and we wanted to abstract away all of the details of these protocols from application developers. We imagined the user experience to be a single one-line function call to create an end-to-end authenticated and encrypted secure channel.

Cryptography related code also tends to have a lot of footguns, one little misstep and your system becomes insecure. So simplicity isn't just an aesthetic ideal for us, we think it's a crucial requirement to ensure that we can empower everyone to build secure systems. Knowing the nitty-gritty of every protocol involved should not be necessary. We wanted to hide these footguns away and provide developer interfaces that are easy to use correctly and impossible/difficult to use in a way that will shoot your application in the foot.

That’s where C was severely lacking.

Our attempts at exposing safe and simple interfaces, in C, were not successful. In every iteration, we found that application developers would need to know too much detail about protocol state and state transitions.

The Elixir Prototype

Around that time I wrote a prototype of creating an Ockam Secure Channel over Ockam Routing in Elixir.

Elixir programs run on BEAM, the Erlang Virtual Machine. BEAM provides Erlang Processes. Erlang Processes are lightweight stateful concurrent actors. Since actors can run concurrently while maintaining internal state, it became easy to run a concurrent stack of stateful protocols - Ockam + Ockam + Ockam .

I was able to hide all the stateful layers and create a simple one line function that someone can call to create an end-to-end encrypted secure channel over a variety of multi-hop, multi-protocol routes.

{:ok, channel} = Ockam.SecureChannel.create(route, vault, keypair)

An application developer would invoke this simple function and multiple concurrent actors would run the underlying layers of stateful protocols. The function would return when the channel is established or if there is an error. This is exactly what we wanted in our interface.

But Elixir isn’t like C. It doesn’t run that well on small/constrained computers and it's not a good choice for being wrapped in language-specific idiomatic wrappers.

Benefits of Rust

At this point we knew we wanted to implement lightweight actors but we also knew C would not make that easy. This is when I started digging into Rust and very quickly encountered a few things that made Rust very attractive:

Rust has compatibility with the C calling convention

Rust libraries can export an interface that is compatible with C's calling convention. Which means that any language or runtime that can statically or dynamically link and call functions in a C library can also link and call functions in a Rust library - in the exact same way. Since most languages support native functions in C, they also already support native functions in Rust. This made Rust equal to C from the perspective of our requirement of having language specific wrappers around our core library.

Rust has support for lots of targets

Rust compiles using LLVM which means that it can target a very large number of computers. This set is likely not as big as everything that C can target with GCC and various proprietary GCC forks but is still a very large subset and there’s work ongoing to make Rust compile with GCC. With growing support of new LLVM targets and potential GCC support in Rust, it seemed like a good bet from the perspective of our requirement of being able to run everywhere.

Rust has strong typing and a powerful type system

Rust’s type system allows us to turn invariants into compile-time errors. This reduces the set of possible mistakes that can be shipped to production by making them easier to catch at development time. Our team and the user of our Rust library become less likely to ship behavioral bugs or security vulnerabilities to production.

Rust has memory safety and the borrow checker

Rust’s memory safety features eliminate the possibility of use-after-frees, double frees, overflows, out-of-bounds access, data races and many other common mistakes that is known to cause 60-70% of high-severity vulnerabilities in large C or C++ codebases. Rust provides this safety at compile time without incurring the performance costs of safely managing memory at runtime using a garbage collector. This gives Rust a serious advantage to write code that needs to be highly performant, run in constrained environments, and be highly secure.

Rust has Async/await and pluggable async runtimes

The final piece that convinced me that Rust is a great fit for Ockam was async/await.

We had already identified that we need lightweight actors to create a simple and safe interface Ockam's stack of protocols. async/await meant that a lot of the hard work to create actors had already been done in projects like tokio and async-std. We could build Ockam's actor implementation on this foundation.

Another important aspect that stood out was that async/await in rust has one significant difference from async/await in other languages like Javascript.

In Javascript a browser engine or nodejs picks the way it will run async functions. But in Rust you can plugin a mechanism of your own choice. These are called async runtimes - tokio is a popular example of such a runtime that is optimized for highly scalable systems. But we don't always have to use tokio, we can instead chose an async runtime optimized for tiny embedded devices or microcontrollers.

This meant that Ockam's actor implementation, which we later called Ockam , if we base it on Rust's async/await can present exactly the same interface to our users regardless of where it is running - big computers or tiny computers. All our protocol interfaces that would sit on top of Ockam Workers can also present the exact same simple interface - regardless of where they are running.

At this point we were convinced we should re-write Ockam in Rust.

In the conversation, that I mentioned earlier, Paul Dix and I discussed what the transition looked like for our teams at Ockam and InfluxDB after each project had decided to switch to Rust. We discussed how InfluxDB moved from Go to Rust and how Ockam moved from C to Rust. In case you're interested, in that part of our journey go listen to the .

Many iterations later, anyone can now use the Ockam crate in rust to create an end-to-end encrypted and mutually authenticated secure channel with a simple function call.

Here’s that one single line, we had imagined when we started:

let channel = node.create_secure_channel(&identity, route, options).await?;

It creates an over arbitrary multi-hop, multi-protocol routes that can span across private networks and clouds. We are able to hide all the underlying complexity and footguns behind this simple and safe function call. The code remains the same regardless of how you use it - on scalable servers or tiny microcontrollers.

To learn more checkout Ockam on Github or try the step-by-step walk throughs of the

Credentials

An Ockam Credential is a signed attestation by an Issuer about the Attributes of Subject. The Issuer and Subject are both Ockam Identities. Attributes is a list of name and value pairs.

Issuing Credentials

Any Ockam Identity can issue credentials about another Ockam Identity.

The Issuer can include specific attributes in the attestation:

Verifying Credentials

Storing Credentials

Trust Anchors

Trust and authorization decisions must be anchored in some pre-existing knowledge.

Anchoring Trust in an Access Control List (ACL) of Identifiers

In the previous section about Ockam we ran an example of using pre-existing knowledge of Ockam . In this example n1 knows i2 and n2 know i1:

Anchoring Trust in a Credential Issuer

Managed Authorities

At Ockam's core is a collection of cryptographic and messaging protocols. These protocols make it possible to create private and secure by design applications that provide end-to-end application layer trusted data.

Ockam is designed to make these powerful protocols easy and safe to use in any application environment – from highly scalable cloud services to tiny battery operated microcontroller based devices.

However, many of these protocols require multiple steps and have complicated internal state that must be managed with care. It can be quite challenging to make them simple to use, secure, and platform independent.

Ockam , , and help hide this complexity and decouple from the host environment - to provide simple interfaces for stateful and asynchronous message-based protocols.

In this post I'm going to show you how to provide more granular and more secure connectivity to and from a SaaS platform. The end result is a holistic solution that looks and feels like a natural extension of the SaaS platform, and is either offered as a feature for enterprise focused plans, or as a competitive differentiator to all your customers. The total time required to run the demo is just a few minutes. I'll also dig deep into what's happening behind the scenes to explain how the magic works.

First, let me give some background on why this specific need arises and highlight the shortcomings in traditional implementations. Because those old approaches don't work any more.

You need to start thinking of security as a feature. If you're a VP of engineering, if you're a product manager, product owner, give time to security, let your developers create a better, more secure infrastructure. — Joel Spolsky, Founder of Stack Overflow

The most successful products over the coming decade will be the ones that realise the status-quo approaches are no longer good enough. You don't need to take Joel's word for it either, take a read of the details of the recently announced from Apple. One of the most successful companies over the past two decades is making a clear statement that security, privacy, and trust will be a core differentiator. They even discuss how current usage of protocols like TLS can't provide the end-to-end security and privacy guarantees customers should expect.

I worked on connecting systems to each other many years ago, a labor-intensive task in the earliest stages of my career. Our company was growing and we'd patch the server room in the current building to the system we just installed in the new building. The new office was a few blocks down the street and we were working with the local telco to install a dedicated line. At the time, connecting two separate networks had an obvious and physically tangible reality to it.

This guide contains instructions to launch within AWS environment, an

• Ockam Outlet Node

• Ockam Inlet Node

The walkthrough demonstrates running both outlet and inlet nodes and verify communication between them.

Read: “How does Ockam work?” to learn about end-to-end trust establishment.

Create an Orchestrator Project

and pick a subscription plan through the guided workflow on Ockam.io.

Run the following commands to install Ockam Command and enroll with the Ockam Orchestrator.

Completing this step creates a Project in Ockam Orchestrator.

Control which identities are allowed to enroll themselves into your project by issuing unique one-time use enrollment tickets. Generate two enrollment tickets, one for the Outlet and one for the Inlet.

Generate enrollment tickets

Setup Ockam Outlet Node

• Login to AWS Account you would like to use

• Subscribe to " in AWS Marketplace

• Navigate to AWS Marketplace -> Manage subscriptions. Select Ockam - Node from the list of subscriptions. Select Actions-> Launch Cloudformation stack

• Click Next to launch the CloudFormation run.

• A successful CloudFormation stack run configures the Ockam outlet node on an EC2 machine.

• EC2 machine mounts an EFS volume created in the same subnet. Ockam state is stored in the EFS volume.

• Connect to the EC2 machine via AWS Session Manager. To view the log file, run sudo cat /var/log/cloud-init-output.log.

Set up a webhook on the ec2 machine to validate connectivity

• Run python3 /opt/webhook_receiver.py to start the webhook that will listen on port 7777. We will send traffic to this webhook after inlet is setup, so keep the terminal window open.

Setup Ockam Inlet Node

• Login to AWS Account you would like to use

• Subscribe to " in AWS Marketplace

• Navigate to AWS Marketplace -> Manage subscriptions. Select Ockam - Node from the list of subscriptions. Select Actions-> Launch Cloudformation stack

• Click Next to launch the CloudFormation run.

• A successful CloudFormation stack run configures the Ockam inlet node on an EC2 machine.

• EC2 machine mounts an EFS volume created in the same subnet. Ockam state is stored in the EFS volume.

• Connect to the EC2 machine via AWS Session Manager. To view the log file, run sudo cat /var/log/cloud-init-output.log.

Validate Connectivity

• Connect to the EC2 machine via AWS Session Manager.

• Run the command below to post a request to the Inlet address. You must receive a response. Verify that the request reaches the webhook running on the Outlet machine.

A Successful setup receives a response back

You will also see the request received in the Outlet EC2 machine

You have now successfully created an Ockam Portal and verified secure communication 🎉.

Cleanup

• Delete the example-outletCloudFormation stack from the AWS Account.

• Delete the example-inlet CloudFormation stack from the AWS Account.

• Delete ockam configuration files from the machine that the administrator used to generate enrollment tickets.

Matthew Gregory: Welcome to the podcast. Every week Mrinal, Glenn, and I get together to discuss technology, building products, secure by design, how Ockam works, and a lot more. We comment on industry macros across cloud, open source, and the security space. That brings us to what we're going to talk about today, which is that often people seem to have a qualification around the importance of their data and what they should be doing with it. They usually qualify the privacy and security of their data, and then build a posture or governance model around that. They talk about different technologies, whether something is encrypted or needs to be encrypted, or how they're encrypting it. I want to unpack that. Let's start with the first one, which is when people qualify the importance of their data and how they need to protect it or what they need to do to secure it, or whether they're encrypting it or not. A lot is that people think their data is far less important than it is and undervalue what happens when that data is used in their applications. Because those applications rely on that data to be truthful and to have integrity to do things with the data. With that, I'll kick it over to Glenn. “My data is not important,” why does this not make any sense?

Your Data is More Important than you Think

Glenn Gillen: My initial reaction is to ask, why are you collecting it then? If the data's not important, just stop doing that. Stop wasting cycles. I think people mean something else when they say that, the word important is probably the wrong one to use. The data might not be commercially sensitive, for example. It could be an aggregated metric stuff or it's public data that anyone could capture. It's not specific to your business or is public. If someone saw that data, you wouldn’t care. Often that is what people mean, they don't care if it's private, rather than the data is not important.

Mrinal Wadhwa: An interesting nuance is that there is the importance axis, and also what is considered data. Oftentimes people will think about what they are collecting as data, but they won’t think about the request to collect that data as data itself. The importance is in the fact that I'm observing some piece of information, collecting it, and then delivering it to a data store. However, the message that carries your data from the place you collected it to the place you store it is also data that is relevant. And that message is relevant to why you are collecting the information. Maybe you're building an AI model, where the data must always be correct when it's fed into that model. It's important because it's critical to whatever your business use case is that relies on that model.

Glenn Gillen: If you asked someone, “What if you replace your data with a random number generator.” Their reaction will be no, the information must be correct. Well, then your data is important. I think that's the best litmus test for what important means. If your data was wrong, would you care? The answer's always yes.

Matthew Gregory:

You're building an application that is going to consume this data and make decisions based on the data to produce some output. It's the process that you have built that's going to consume and transform the data. That is the important part. Even if the data is public, for example, we spoke to a wind farm operator who claimed their data was not important. It’s wind speed data, which is public. But they're making important business decisions, such as turning on and off the wind farm, and making forecasts on how much energy is being produced, based on that data. The applications in the data center need accurate information. So ergo the data is important. The data is important and the integrity of that data is critical because your application depends on it. Another thing we hear a lot is about the importance of data privacy. I think this goes actually in two directions. People either over-index on it or don't index on it. I think both are interesting to talk about. Privacy of data is a big topic right now, just in the press, with the new SEC rule where you have to disclose a data breach if you're a publicly traded company. Mrinal, if someone says, “My data doesn't need to be kept private.” What would be the other concerns that we would think about when we're talking about privacy of data? Even if we don't care that it leaked outside of our data center, what else should you be thinking about when you're thinking about the privacy of data?

Security is (alot) more than just Privacy

Mrinal Wadhwa: Let's take the case of “my data doesn’t need to be kept secret.” It's okay if everybody knows that I collected some information. What's usually not okay is the information I collected is incorrect. If that is important to you, then the tools that give the data properties of data integrity are important to you. So it might be okay that the data is revealed to someone, but you probably are not okay with the data being tampered, or the data not coming from the right source. So even when you don't care about the privacy, the secrecy, or the confidentiality of a piece of information, you still care about the integrity and authenticity of that information.

Matthew Gregory: Let's break this down to a specific example. There's some application living on the internet that's producing data, originating it, and it sends the data across the internet to a database that's going to store it before it eventually into an application where it will be processed. In this data producer-to-data-store scenario, we may not care if a malicious actor can see the data. But we do care about the integrity, authenticity, and originality of that data, why is that?

What can happen without Data Integrity

Mrinal Wadhwa: Let's say the message to the database is, “Write to the readings table that the observation is 500.” That's the message that's going over the wire, and we've already decided that the reading of 500 is not private information. But we still care that when that message reaches the data store, the value of 500 is in fact written to the readings table. Those two pieces of information, that it's being written to the right table and the value is 500, must remain the same from when the message started to when the message was received. We also care that only the authenticated data source can write to the reading table. We don’t want an attacker in the middle to be able to generate random messages that get stored in our database. It'll bump up our database bill, it will create garbage in our database that will affect our AI that learns on this database.If that happens, garbage gets fed into our system or we end up spending a lot of time and money dealing with the data. Both of those would be bad, and it just doesn't end there. The data store will acknowledge that it received the data, and an attacker in the middle could block that acknowledgment or incorrectly send it (i.e. send an acknowledgement when an action didn't happen).That would be bad too. If I can block the acknowledgment, the source keeps sending the same data over and over again. This has the same effect, many readings get created instead of one because I was able to block the acknowledgment. This results in more data getting stored in the database, or garbage getting stored. An attacker can also send incorrect acknowledgments. Even though the source said, “store the reading 500”, that reading never gets stored because the attacker in the middle sends a fake acknowledgment. What this means is, that even though we didn't care about the privacy of this flow of information, we still care about making sure only the correct source can write to the data store. We care that only the data store can acknowledge the fact that the data has been successfully written. So we care about the authenticity of these messages and the integrity of these messages because we don't want incorrect readings to be stored or stored in the wrong table.

Matthew Gregory: That's right. And to unpack that even further, if the message is in clear text when it goes over the wire, an attacker in the middle could know how to write to the database. They could perform man-in-the-middle attacks. Because they know what's happening between the application and data store, where the data came from, where it's going, what the message is, and how to write to it. It would be even worse if you have an unprotected credential in there as well. The punchline here is that privacy is not the only security concern, which brings us to unpacking ‘security’ as a word. And how people go about setting up security, thinking about data governance, and control of data. Glenn, could you talk about the spectrum of how people think about secure systems, in this distributed computing world, compared to the simpler days when our applications were all in the same box?

Glenn Gillen: I think that, as an industry, we don't care about privacy enough. The SEC ruling is forcing companies to think about it. But that said, when we start to think about security, we myopically think about privacy more often than not. There's a juxtaposition of, we're not thinking about privacy enough, but when we do think about it, we think about it too much at the expense of integrity and authenticity. Matt Johansson gave an example of how people think of security like visiting an emergency room. Something happens, you triage it, and you're done and you move on. What we witness time and time again is, that's not the case. Attackers get in and they lay low for a long time. These little things escalate over time. They'll sit there and they'll do some reconnaissance on your systems. His point was that security is more like a mental health condition. It's something that needs constant nurturing, and essentially therapy. You need to maintain it and keep yourself in a healthy state. It's not something you can just triage. People tend to think about it from a privacy perspective, at the expense of all the other things. Someone getting into your system and exposing your data is embarrassing and that's why it's human for us to focus on privacy. You'll get called out on it publicly. You impact a lot of people. The idea of someone getting in and quietly polluting your data for an extended period of time is quite terrifying to me these days. As an industry, we're a little bit asleep at the wheel in helping people understand those risks, especially if you're feeding data into an AI model. You're training a model on all of the data that you've been collecting for years and making business decisions based on it. If you can't absolutely trust all of the inputs into that system, the outputs will be garbage. We talked about the wind farm example before. That's a triage-type problem. If you get a wind speed rating of 900 knots and you've mistakenly shut down a turbine because it can't operate at that speed, you know immediately that a mistake has been made. If an attacker was instead quietly polluting the data with random variance, that data would be gone. You can't go back and fix it, you don’t know when the problem emerged, and you can't clean it. Your data is forever polluted. Your business intelligence is ruined. People think about security as just privacy, and they've conflated it for a long time because of the emotional attachment to privacy. There's a much bigger business risk looming, in my opinion, around integrity, authenticity, and control of the data.

Mrinal Wadhwa: Think about the amount of investment that goes into training an AI model, all of the data, and GPUs. If that data has been polluted, that can't be unrolled anymore. It's part of the model you trained and spent a lot of compute and storage training, for years. All of that becomes a wasted investment, or worse the results impact the action you take based on the model. There are also immediate strategic impacts. If the messages are being actively tampered with and there's an attacker somewhere in your environment, they can selectively tamper messages that are supposed to indicate various actions, such as delete a database or spin up a bunch of instances, all of those are pieces of data. If you don’t have guarantees around authenticity and integrity, those messages can cause unwanted action.

Why you need encryption (even when you don’t think you do)

Matthew Gregory: Let's unpack this, Mrinal. When someone says, “I don't need encryption.” We established that they should care about integrity and authenticity, so why do they also need encryption?

Mrinal Wadhwa: I think “I don't need to encrypt my data” usually comes from a place of, “I don't think my data needs to be private or secret,” and it's never because people don’t think their data needs to be correct. So people care about the correctness of this information. If you care about the integrity of the data, well it turns out that the mechanisms to have privacy, integrity, and authenticity guarantees work together. In Ockam’s case, Ockam secure channels provide those guarantees together. It’s more expensive and often less secure to decouple these properties, so they come as a package. You get data integrity, authenticity, and confidentiality as a package. When people are saying they don't want encryption, they are saying that it is okay if the data isn't kept secret or private, but they're not saying my data should not have data integrity and authenticity. Since they care about that, they need encryption because that’s the way you get a guarantee of integrity and authenticity.

Glenn Gillen: The flip side of that is someone whose data is already encrypted. From my experience as a web developer, often you’ll hear, “I'm using TLS, I've got TLS to the API.” The question is, what do you actually have there? You have a guarantee around privacy that only the server can read that information. That's what most TLS setups give you. But for a lot of apps, TLS terminates at the CDN because you don't want it to travel. You end up with a privacy guarantee for the length of that TLS connection to the CDN, and often ignore everything that happens behind the scenes and trust that your providers and intermediaries are not looking at the data and have put controls in place. And that doesn’t answer the question of integrity. Because all you have is privacy, and then we're back to Mrinal’s point; you care that it's correct and you want integrity guarantees. Then you need to verify who is sending the data. You could do mutual TLS, but often people are not doing that for all of their clients all of the time. And if you are, now you have to manage keys. So very quickly you're in a place where you think you have privacy, but it's a really small definition of privacy. You don’t have any guarantees around integrity.

Mrinal Wadhwa: Security is often defined from the perspective of whoever is controlling a certain system or is responsible for a certain asset. Let's say I'm the person responsible for making sure the data stored in that AI database is always correct. If that's the responsibility, I need to have control over who gets to store data in my database, and what they get to store. You can only have that control if you have the ability to know who sent a piece of information and whether what they sent is exactly what they sent. Was it tampered with along the path? You need control over the behavior of your system, what information is stored in it, et cetera to have that control.

You need authenticity of who is sending requests and who you are sending responses to. You need an integrity guarantee on what the requests and responses are. Since end-to-end encryption brings these properties together, I also get privacy guarantees. In some cases, you may care about privacy, in others you might not. But if you need your system to be secure, you need a tool that has these properties and they tend to come as a package.

Encryption is not Security

Matthew Gregory: It’s often a trap to think you are secure because your data is encrypted. The question is, where are we talking about? Is it data at rest in your database or while it's moving? Are you talking about a single TLS connection? Glenn, could you talk about this false narrative that having encryption means you are secure and that your data is private?

Glenn Gillen: TLS is an easy example to pick on because it feels like people have been given a checklist of stuff to go through. You have encryption in transit, encryption at rest, job done. Right? Let’s say I have a TLS connection from my desktop here to a pop in Melbourne. What happens behind the scenes? It’s the shared responsibility model, you speak to a vendor, look at their SOC2, compliance reports, and make sure it’s been audited. If anything in that supply chain gets compromised, you don't know what has been exposed and you don’t have control. When you outsource control to a cloud vendor, ultimately you are still responsible for security. You can have shared responsibility for it, but I think the easiest way to solve this is to focus on the two ends of the system that you can control. If you can control both of those ends of the systems, you can build solutions that are secure all the way through, no matter what that supply chain looks like, no matter what the network or the topology is. You can build highly trustful systems in that environment. The same thing goes for a managed service or message queue. Most of them are designed, intentionally, to have the data available in plain text through that system. It's not just a pipe, they're trying to provide analytics and build tooling around that. It’s part of the business model in a lot of those cases to have the data be plain text and visible in the system. That’s not what you want. You want integrity, control, and privacy all the way through your system. So you can tick a box and say, “We've got TLS.” But then you ignore the fact that it's plain text during this important, high-value moment. And then it's encrypted again on the way back out. That’s not private.

The Key to getting Encryption Right

Matthew Gregory: When we're talking to people about their architecture and encryption, the first thing we jump to is the keys. How and where were they created? How were they distributed? If you're generating symmetric keys for your encryption and sending them out into the world or building them into software, now you have a vulnerability around how the key was created and how it was distributed.

Mrinal Wadhwa: If someone's building a system and all the participants in that system have one key that stays the same forever, and you call an AES function to encrypt using that one key, you effectively have no encryption. It might feel like it, but it's not encryption. Calling AES is not encryption. Encryption needs to be end-to-end and needs to have a series of properties that have been studied for about 30 years now.

There are a ton of mistakes that can be made along the way. You have to think about the rotation of keys, there's a bunch of work around ratcheting keys, getting forward secrecy properties, getting properties that prevent impersonation, and replay attacks.

There’s a host of problems that come with managing keys. A single key everywhere is not the answer, but it's easy to convince yourself you did enough and you did it right. There are so many CVEs in the history of secure channel designs that are all about people thinking they did encryption correctly but made a mistake. So that's why a lot of these properties are now proven using formal models. For example, in Ockam’s secure channel design, we have formal proofs of various properties in our design. So there's a lot that goes into doing key management well.

Matthew Gregory: Yeah, it is the first question that gets asked after encryption. You said the data is encrypted, now we have to have a whole conversation about what that actually means. You may have done some things well, but that doesn't mean you have security, privacy, integrity, and authenticity. All these things go together. Using proven techniques is very important to build an architecture that is secure by design and has a low vulnerability surface. It's pretty difficult to do ad hoc. With that, I'll wrap up this podcast. That was a little insight into some of the things that the three of us talk about, hopefully, that was helpful. More to come and we'll see you later.

This guide contains instructions to launch within AWS environment, an

• An Ockam Postgres Outlet Node within an AWS environment

• An Ockam Postgres Inlet Node:

  • Within an AWS environment, or

is a fully managed service that makes high-performing foundation models (FMs) from leading AI companies and Amazon available for your use through a unified API. Organizations building innovative generative AI applications with Amazon Bedrock often need to ensure their proprietary data remains secure and private while accessing these powerful models.

By default, You can access Amazon Bedrock over the public internet, which means:

1. Your API calls to Bedrock travel across the public internet.

2. Your client must have public internet connectivity

Now that we understand the basics of Nodes, Workers, and Routing ... let's create our first encrypted secure channel.

Establishing a secure channel requires establishing a shared secret key between the two entities that wish to communicate securely. This is usually achieved using a cryptographic key agreement protocol to safely derive a shared secret without transporting it over the network.

Running such protocols requires a stateful exchange of multiple messages and having a worker and routing system allows Ockam to hide the complexity of creating and maintaining a secure channel behind two simple functions:

• create_secure_channel_listener(...) which waits for requests to create a secure channel.

At Ockam's core is a collection of cryptographic and messaging protocols. These protocols enable private and secure by design applications that provide end-to-end application layer trust in data.

Ockam is designed to make these protocols easy and safe to use in any application environment – from highly scalable cloud services to tiny battery operated microcontroller based devices.

Many included protocols require multiple steps and have complicated internal state that must be managed with care. Protocol steps can often be initiated by any participant so it can be quite challenging to make these protocols simple to use, secure, and platform independent.

Ockam , , and help hide this complexity to provide simple interfaces for stateful and asynchronous message-based protocols.

Transcript

Matthew Gregory: We describe Ockam as Networkless. One of the interesting things about Ockam is that we enable security and trust at the application layer, between applications that are in distributed locations. What you also get from this are all things you don’t have to do because you use Ockam, and a lot of those things are at the network layer. This podcast is dedicated to the topic of Networkless. Maybe it will be controversial. Maybe everyone will get it.

Networkless: a new abstraction layer

We've all suffered through serverless and all the heads banging on the table from that. but the analogy is very similar. Obviously, there is a network involved in moving data between applications and remote networks. Ockam is using those networks, but the key thing is that the end user of Ockam and the application developer that's trying to access data doesn't have to think about all of these network layer problems, which is often where you can make a mistake and have a data leak or a privacy issue or some sort of security vulnerability. Many times in big organizations, the person who's developing the application that needs to access the data doesn't have the capacity or capability to change the network. And they might not even know who the network team is. They're just focused on their application. So how do we empower them to build applications that can access data in a trustful way? Glenn originated the Networkless idea when we started talking about Ockam together. I'm curious how you came up with this analogy between Ockam and the idea of Networkless and how you thought this was so similar to Serverless. What do you think, Glenn?

This guide contains instructions to launch within AWS environment, an

• An Ockam Redshift Outlet Node within an AWS environment

• An Ockam Redshift Inlet Node:

  • Within an AWS environment, or

  • Using Docker in any environment

The walkthrough demonstrates:

1. Running an Ockam Redshift Outlet node in your AWS environment that contains a private Amazon Redshift Serverless or Amazon Redshift Provisioned Cluster

2. Setting up Ockam Redshift inlet nodes using either AWS or Docker from any location.

3. Verifying secure communication between Redshift clients and Amazon Redshift Database.

Read: “” to learn about end-to-end trust establishment.

PreRequisite

• A private Amazon Redshift Database (Serverless or Provisioned) is created and accessible from the VPC and Subnet where the Ockam Node will be launched.

• Security Group associated with the Amazon Redshift Database allows inbound traffic on the required default port (5439) from the subnet where the Ockam Outlet Node will reside.

• You have permission to subscribe and launch Cloudformation stack from AWS Marketplace on the AWS Account running Amazon Redshift.

Create an Orchestrator Project

1. and pick a subscription plan through the guided workflow on Ockam.io.

2. Run the following commands to install Ockam Command and enroll with the Ockam Orchestrator.

1. Control which identities are allowed to enroll themselves into your project by issuing unique one-time use enrollment tickets. Generate two enrollment tickets, one for the Outlet and one for the Inlet.

Setup Ockam Redshift Outlet Node

• Login to AWS Account you would like to use

• Subscribe to "Ockam - Node for Amazon Redshift" in AWS Marketplace

• Navigate to AWS Marketplace -> Manage subscriptions. Select Ockam - Node for Amazon Redshift from the list of subscriptions. Select Actions-> Launch Cloudformation stack

• Click Next to launch the CloudFormation run.

• A successful CloudFormation stack run configures the Ockam Redshift Outlet node on an EC2 machine.

  • EC2 machine mounts an EFS volume created in the same subnet. Ockam state is stored in the EFS volume.

  • A security group with egress access to the internet will be attached to the EC2 machine.

Ockam redshift outlet node setup is complete. You can now create Ockam redshisft inlet nodes in any network to establish secure communication.

Setup Ockam Inlet Node

You can set up an Ockam Redshift Inlet Node either in AWS or locally using Docker. Here are both options:

Option 1: Setup Inlet Node in AWS

• Login to AWS Account you would like to use

• Subscribe to " in AWS Marketplace

• Navigate to AWS Marketplace -> Manage subscriptions. Select Ockam - Node from the list of subscriptions. Select Actions-> Launch Cloudformation stack

• Click Next to launch the CloudFormation run.

• A successful CloudFormation stack run configures the Ockam inlet node on an EC2 machine.

• EC2 machine mounts an EFS volume created in the same subnet. Ockam state is stored in the EFS volume.

• Connect to the EC2 machine via AWS Session Manager.

Use any postgresqlclient and connect to localhost:15432 (PGHOST=localhost, PGPORT=15439) from the machine running the Ockam redshift Inlet node.

Option 2: Setup Inlet Node Locally with Docker Compose

To set up an Inlet Node locally and interact with it outside of AWS, use Docker Compose.

• Create a file named docker-compose.yml with the following content:

• Run the following command from the same location as the docker-compose.yml and the inlet.ticket to create an Ockam postgres inlet that can connect to the outlet running in AWS , along with psql client container.

• Check status of Ockam inlet node. You will see The node is UP when ockam is configured successfully and ready to accept connection

• Connect to psql-client container and run commands

This setup allows you to run an Ockam Redshift Inlet Node locally and communicate securely with a private Amazon Redshift database running in AWS

• Cleanup

Attribute names can be used to define policies and policies can be used to define access controls:

• Policies are expressions involving attribute names, which can be evaluated to true or false given an environment containing attribute values.

• Access controls were discussed earlier. They restrict the messages which can be received or sent by a worker.

Policies

Policies are boolean expressions constructed using attribute names. For example:

In the expression above:

• and, =, member? are operators.

• resource.version, subject.name, resource.admins are identifiers.

Values can have the 5 following types:

• String

• Int

• Float

This table lists all the available operators:

Examples

Here are a few more examples of policies.

The subject must have a component attribute with a value that is either web or database:

Note that attribute names can have dots in their name, so you could also write:

You can also declare more complex logical expressions, by nesting and and or operators:

The subject must either by the "Smart Factory" application or being a member of the "Field Engineering" department in San Francisco:

Boolean policies

Since many policies are just need to test for the presence of an attribute, we provide simpler ways to write them.

For example we can write:

Simply as (note that logical operators can now be written as infix operators):

String comparisons are still supported, so you could also have a component attribute and write:

More complex expressions require parentheses:

Since identities are frequently used in policies, we provide a shortcut for them. For example, this is a valid boolean policy:

It translates to:

This table summarizes the elements you can use in a simple boolean policy:

Evaluation

We evaluate a policy by doing the following:

• Each attribute attribute_name/attribute_value is added to the environment as an identifier subject.attribute_name associated to the value attribute_value (always as a String). In the example of a policy given above the identifier subject.name means that we are expecting an attribute name associated to the identity which sent a message.

• The top-level expression of the policy is recursively evaluated by evaluating each operator and taking values from the environment when an expression is referencing an identifier.

Access controls

The library offers two types of access controls using policies:

1. AbacAccessControl.

2. PolicyAccessControl.

AbacAccessControl

This access control type is used as an IncomingAccessControl (so it restricts incoming messages).

We define an AbacAccessControl with the following:

1. A Policy which specifies which attributes are required for a given identity.

2. An IdentityRepository which stores a list of the known authenticated attributes for a given identity.

When a LocalMessage arrives to a worker using such an incoming access control, we do the following:

• If an identity is not associated to this message (as LocalInfo), the message is rejected.

• Otherwise the attributes for this identity are retrieved from the repository.

• The attributes are used to populate the policy environment.

PolicyAccessControl

This access control type is used as an IncomingAccessControl (so it restricts incoming messages).

We define a PolicyAccessControl with the following:

• A PolicyRepository which stores a list of policies.

• A Resource and an Action. They represent the access which we want to restrict.

• An IdentityRepository which stores a list of the known authenticated attributes for a given identity.

When a LocalMessage arrives to a worker using this type of incoming access control, we do the following:

• If an identity is not associated to this message (as LocalInfo), the message is rejected.

• Otherwise the attributes for this identity are retrieved from the repository.

• The most recent policy for the resource and the action is retrieved from the policy repository.

The two major differences between this policy and the previous one are:

1. The PolicyAccessControl models a Resource/Action pair.

2. Policies for that resource and action can be modified even if the worker they are attached to is already started.

Transcript

Matthew Gregory: Welcome to the Ockam podcast. Every week, Mrinal, Glenn, and I will get together to discuss technology, building products, security by design, how Ockam works, and a lot more. We'll also comment on industry macros from across cloud, open source, and security. We also plan on bringing guests to add some perspective and challenge some of the dogmas that we or others might have.

Today we'll spend some time allowing you to get to know us, where we come from, and what motivates us to build Ockam: the company, the product, the team. And with that, Mrinal could you start us off with what you've been up to in your professional career and what led you to what we're doing here at Ockam?

The challenge in building Trustful systems

Mrinal Wadhwa:

Sure. Thanks, Matt. My name's Mrinal, I am CTO at Ockam and my background has been in distributed systems. I started my career working on large-scale data problems with tools like Erlang, Hadoop, etc. dealing with large amounts of streaming data. And then about 10 years ago I took on a role as CTO of a hardware business, which was strange because I had no hardware background.

It was interesting because it was a great team that wanted to turn the hardware they'd been building into a set of products of connected systems for city infrastructure. And so it's essentially IoT that was installed inside cities, airports, factories, and things like that.

And in designing that very large distributed system, we thought a lot about how we could trust the information that was flowing through that system. How do we trust a sensor sitting in a city street telling us something? Or how do we securely deliver a software update to a device installed inside a factory?

Because this was 2013/14, there weren't other systems like this that had done this at scale. There were no off-the-shelf IoT platforms you could buy from anyone. So, we were thinking about a lot of problems from scratch with no reference point.

We thought about the security and trust problem extensively. That led us to design protocols in that system to do authentication of devices, authentication of the update infrastructure, secure delivery of software updates, secure delivery of the data that was coming in from the sensors, and trustful control of things that are distributed out in the world.

A few years later, as IoT became more prevalent in the world, I noticed that a lot of people were struggling with the security problems around IoT. What I realized was that because of the type of product we were building and the type of customers we were going after, we had invested the time and energy to secure our infrastructure, which meant building a lot of stuff from scratch. And it took us a few years to get it to a secure point. However, for everyone else in the IoT market, that wasn't their core focus. They were trying to solve other problems and did not focus on security.

And what ended up happening is that IoT targets became more attractive to attack. A lot of people attacked it. I remember in 2016/17 there was this big Marai botnet incident, which brought attention to this set of problems. So my realization was, that as systems become more and more distributed, the problem of trust, security, and reliable communication becomes harder and harder.

You have to think about managing keys at scale, you have to set up secure protocols that traverse various types of networks, et cetera. And building all of that takes a lot of time, expertise, and money, which not everybody could invest. And so I started thinking about something that should exist off the shelf to solve that set of problems.

And that led me to meet Matt around 2018. In our very first conversation, what we connected on was the idea that the problems I was thinking about in IoT are more general because all systems are becoming more and more distributed. As that happens, you have to do cross-cloud communication, cross-company communication, cross-network communication, and all of those scenarios. Trust, security, and identity are hard problems. So we both resonated on this idea that a set of tools should exist that make all of this simple for anyone building a distributed system. So that's been my journey.

The evolution of cloud architectures led us to build Ockam

Matthew Gregory: The first meeting we had was a meeting of the minds. We came to this conclusion, it was immediate. It's funny how people could be thinking of the same problems in parallel and come to similar conclusions.

So my background, in the 2000s I was doing IoT-like things, but the term hadn't been created at that point in time. I was building real-time data analytics systems for America's Cup teams, recording what was going on in sailboats, very similar to what happens in Formula One.

If anyone watches the Netflix show on Formula One, you'll see a lot of this in action. It was the same thing in sailing. Then in the late 2000s, I moved from a builder of systems to more of a tool maker. I worked with Weather Underground to build an API for any software developer that needs to access weather data.

If you've seen weather on your phone, it probably came from the Weather Underground API that we built right around the time that the iPhone and App Store came out. A lot of people needed weather data for their apps. Then I met Glenn at Heroku on my next hop on the journey.

And this was very early in the cloud era. I was at one of the very first AWS Re:Invents. Heroku is a massive AWS customer. We provided that abstraction that made it easy for the full-stack developer to merge and deploy things to the cloud.

I went from there to Microsoft, right after Satya took over. He put together a red team to figure out how to pivot Microsoft from a platform as a service to infrastructure as a service that could run anything. And obviously, Microsoft loves Linux. A lot of that came out of the work that our team did.

Through what I saw during that cloud era, there was this trend. In the Heroku days, there was a slider bar on the website, and the more you slid it to the right, the more cloud you got. And you could just keep scaling your app until infinity.

At the time, people were building these Ruby Rail apps. As it turns out, you can't scale an app with the cloud to infinity; it will break. For the people that were around at the time, this was the era of the Twitter fail whale. There were all sorts of apps breaking, particularly as mobile apps were coming online and all the backend systems that were supporting them.

So we entered this cloud era where monolithic applications needed a solution. And the solution to that when I was at Microsoft was partnerships with companies like Docker and Mesosphere. You take a monolithic application, divide it up into little pieces, call them containers, and then we can orchestrate how many underlying resources we give all these little microservices or containers underneath it.

So you need an orchestration layer, like Mesosphere, Zookeeper, and Kubernetes. You do infrastructure as code with things like Terraform. There was an entire class of orchestration tools for managing infrastructure so that you could run thousands of applications and scale them up and down as you needed.

The cool thing about being at Microsoft at that time was that we got to see all these use cases where people were running things on-prem, and also in clouds like Azure and AWS, and consuming services from other third-party data service companies. So it wasn't just that the monoliths were being chopped up into little pieces, but they were also being distributed all over planet Earth.

If you then extend this to what problem next needs to be solved, it is "How are we going to create trust and interoperability between all these different applications?" Because you still have a job to be done. And so that concept of a monolith still exists, but you don't have the reality of it all running in the same box or the same cloud or in the same environment.

How are we going to get interoperability, have trust between applications, and move data between them in a trustful way, so that we can have these massive-scale enterprise applications that are distributed all around the world? And as I said, I met Glenn over 10 years ago now when we were working together at Heroku, and that's where we met. I've mentioned HashiCorp where you spent some time, why don't you give us the background about what brings you to Ockam?

Lessons from Heroku, AWS, and Hashicorp

Glenn Gillen: I started as a developer, doing dotnet, Ruby, PHP, a whole bunch of stuff. But ultimately I think the interesting part of that story is ending up at Heroku, which is more than 15 years ago at this point.

I think people who weren't around at the time underestimated the impact it had on so many things. Git-based deployments of code were something they pioneered. Docker wasn't around then. Containerization and deployment of apps to containers at scale was a thing that Heroku helped bring to the masses.

So Matt and I worked on the add-ons marketplace there, which is the way to connect other things. All the bits that aren't Heroku that you plug into: databases, logging services, caching providers, etc. The things that you can't run yourself on Heroku, that's what the add-ons marketplace did. We spent years there building out a great user experience to make it easy to connect to other things.

After Heroku, I went to AWS for a little while. I was helping companies that were at the forefront of trying to adopt the cloud, especially startups, people that were pushing the envelope. I was helping them be successful on AWS. My experience there was that AWS is great at a lot of things, they built an incredible product. But they don't provide a great developer experience. On top of that, you've got access to all these tools. But you're left to do a lot of the plumbing yourself. Especially coming from Heroku, which focused on providing a great abstraction, that friction was felt deeply day to day. I was getting a bit frustrated with it and couldn't work out how to fix that at a company the size and scale of AWS.

How can you fix that? Terraform seemed like the best place to do it. They had a much better UX than AWS and had better coverage in cloud formation at the time if you wanted to do infrastructure as code. So I managed to get myself a job at HashiCorp, working on Terraform and ultimately was Product Lead for Terraform.

We did pretty well with that, but good things come to an end. And after the IPO I started looking around for what might come next, and that's when I reached out to Matt to see what he was up to. It wasn't meant to be a job conversation. But it just evolved.

When I looked at what you were doing here at Ockam, and compared it to my journey, the consistent thing was that we kept making incremental improvements to the way we connect things. We had a set of tools, we'd make them slightly better and we'd still have the same problems. If Ockam existed when we launched Terraform Cloud, we could have saved over six months of development.

We built our own smaller, more focused, less functional version of Ockam to help connect Terraform Cloud to on-prem things. And that was the other thing I learned from my time at Hashicorp, the world's not as simple as it was back when we started Heroku.

You're not just deploying everything to AWS, there are heterogeneous deployments in terms of multi-cloud or hybrid approaches. You've at least got a data center on-prem still that you're trying to talk to, and it's just messy. And we're still trying to patch this in hacky ways. Or in HashiCorp's case, investing half a year trying to build a solution to fix what should be simple. We're just trying to connect two things securely. I thought, oh, this is where I should be spending my time. This is the next thing to go and improve. So that's ultimately how I ended up here.

Matthew Gregory: Glenn's been with us for a little over a year and it's just been awesome to get the band back together. It begs the question, what is Ockam?

Mrinal, when people say, "I've heard of Ockam, but tell me more about it.". What do you tell them?

What is Ockam? For the technical

Mrinal Wadhwa: I'll take two different approaches, from the top down and from the bottom up. I'll attempt the top-down answer first and we can drill into the details later. Ockam is a tool for a developer to build applications that can trust the data in motion between applications.

It could be distributed parts of the same app, or it could be two apps communicating with each other. We give a developer the tools to add trust to that communication and the data that's moving through. But what does that mean? If you peel a layer below that, what that means is we've made the hard parts; mutual authentication, end-to-end encryption, and granular authorization policies on the data flow, we made all of that really easy to add to an application.

If I have an app in a data center and I want to communicate with another part of that application in AWS cloud, we can make it so that the communication is end-to-end encrypted, mutually authenticated, and has granular authorization policies enforced on it.

If you go another layer below, Ockam is a collection of cryptographic and messaging protocols that are wrapped in a programming library that you can call with very simple one or two-line functions to get these capabilities. You don't have to know the underlying secure channel protocol to establish a secure channel. You just get a function called create secure channel, and it gives you a secure, end-to-end encrypted, mutually authenticated channel. So that's what Ockam does, and I can keep talking about the details of how the layers below work, but that's my answer.

What is Ockam? For the non-technical

Matthew Gregory: I'm looking forward to the episode where we get into the protocol and we go through all the protocol design. We collaborated with Trail of Bits, who did our security audit, on a paper describing all of this. That was a fun project and also a forcing function to lay out in simple terms how it all works.

I describe Ockam as WhatsApp for Enterprise data-in-motion systems.

I use WhatsApp because the way that WhatsApp and Ockam works is pretty similar. The other thing about this metaphor that I like is it emphasizes that this is an application layer solution. It's not at the security layer. So we operate at the application layer, and here's how the story goes with WhatsApp. Let's say, I have WhatsApp on my phone and you have WhatsApp on your phone, these two applications can reach each other through the internet without having to touch any of the underlying infrastructure.

My application can go find Mrinal's application through the internet, without him having to touch anything in his network. These two applications can set up a mutually authenticated connection with each other.

So when I'm texting with Mrinal, I know that Mrinal is the person receiving my message. I know that it's him on the other end. Or essentially the application is a proxy for Mrinal because he's the one driving it through his phone. So now we have an identity, which means that we can do mutual authentication that's exclusive between these two applications, and then we can move messages between each other in an end-to-end encrypted way.

And the unique thing about this is that WhatsApp sits in the middle. My phone is not directly connecting to Mrinal's. It has to go out of the infrastructure that I'm currently in, from this network up to the cloud, then go find the WhatsApp server, leave the WhatsApp server, traverse the internet, make it back into Mrinal's network, and then land on his phone.

So there are all these hops along the way. But the cool thing is because it's end-to-end, there's no intermediary along the way that can have access to this data. It's end-to-end encrypted, and setting that up is really difficult. And specifically, WhatsApp cannot read any of these messages. Even though we have that in this consumer product, it is exceptionally difficult to set up in an enterprise data world.

And a lot of the difficulty is because you're not dealing with a lot of the magic that comes with iPhones and Google phones. So it's very difficult to do such a seemingly simple thing. Ockam is the mutual authentication, end-to-end encryption service for any two applications sitting anywhere inside an enterprise or even between two different enterprises, two different companies that need to connect and share data in a peer-to-peer way. Glenn, how do you describe Ockam?

Enterprise vs. Consumer end-to-end encryption options

Glenn Gillen: Well, in our early conversations before I joined, that analogy was part of the 'aha' moment for me, when I realized the potential of Ockam. In my personal life, 100% of the people that I message regularly are using iMessage, Signal, or WhatsApp. So the consumer experience around trusted, encrypted, private end-to-end messaging is so ubiquitous and you never think about it.

We've all been conditioned to expect that as the norm now. You message someone, you get the blue bubble. I didn't have to worry about what network I was on, or did I open up a port to my firewall. It always just works, everywhere all the time.

And then as you were telling me that story, I thought: If I've got a container or a web process and I'm trying to connect it to a database, if those two things are in the same network, then it's relatively quick and easy. But the moment I put the database in a private subnet and that process is anywhere else, even in the same VPC or another cloud, you have a much bigger challenge.

I have to change security groups, firewalls need to open, and in the best case, it's hours of work and dozens of things I need to do. And if any one of those things goes wrong, I've either accidentally exposed my data, or I don't get it working.

There are so many failure modes there. And that was part of what pulled me over here: why are our tools so poor when the consumer ones are so good? Bringing those two things together and that experience to a similar place is what was most exciting to me about Ockam.

Matthew Gregory: Great. With that, we can wrap up our first episode as I said we could keep it simple with this one. We'll dive into some nitty gritty topics as we go along. Let us know what you'd like to talk about, and we'll see you on the next episode of the Podcast. Have a good day. Bye.

At Ockam’s core are a collection of cryptographic and messaging protocols. These protocols make it possible to create private and secure by design applications that provide end-to-end application layer trust it data.

Ockam is designed to make these powerful protocols easy and safe to use in any application environment, from highly scalable cloud services to tiny battery operated microcontroller based devices.

However, many of these protocols require multiple steps and have complicated internal state that must be managed with care. It can be quite challenging to make them simple to use, secure, and platform independent.

Ockam Nodes and Workers help hide this complexity and decouple from the host environment - to provide simple interfaces for stateful and asynchronous message-based protocols.

Nodes

An Ockam Node is any program that can interact with other Ockam Nodes using various Ockam Protocols like Ockam and Ockam Secure Channels.

Using the Ockam Rust crates, you can easily turn any application into a lightweight Ockam Node. This flexible approach allows your to build secure by design applications that can run efficiently on tiny microcontrollers or scale horizontally in cloud environments.

Rust based Ockam Nodes run very lightweight, concurrent, stateful actors called Ockam . Using Ockam Routing, a node can deliver messages from one worker to another local worker. Using Ockam Transports, nodes can also route messages to workers on other remote nodes.

A node requires an asynchronous runtime to concurrently execute workers. The default Ockam Node implementation in Rust uses tokio, a popular asynchronous runtime in the Rust ecosystem. We also support Ockam Node implementations for various no_std embedded targets.

Create a node

The first thing any Ockam rust program must do is initialize and start an Ockam node. This setup can be done manually but the most convenient way is to use the #[ockam::node] attribute that injects the initialization code. It creates the asynchronous environment, initializes worker management, sets up routing and initializes the node context.

For your new node, create a new file at examples/01-node.rs in your project:

Add the following code to this file:

Here we add the #[ockam::node] attribute to an async main function that receives the node execution context as a parameter and returns ockam::Result which helps make our error reporting better.

As soon as the main function starts, we use ctx.stop() to immediately stop the node that was just started. If we don't add this line, the node will run forever.

To run the node program:

This will download various dependencies, compile and then run our code. When it runs, you'll see colorized output showing that the node starts up and then shuts down immediately 🎉.

Workers

Ockam run very lightweight, concurrent, and stateful actors called Ockam Workers.

When a worker is started on a node, it is given one or more addresses. The node maintains a mailbox for each address and whenever a message arrives for a specific address it delivers that message to the corresponding registered worker.

Workers can handle messages from other workers running on the same or a different node. In response to a message, an worker can: make local decisions, change its internal state, create more workers, or send more messages to other workers running on the same or a different node.

Above we've , now let's create a new worker, send it a message, and receive a reply.

Echoer worker

To create a worker, we create a struct that can optionally have some fields to store the worker's internal state. If the worker is stateless, it can be defined as a field-less unit struct.

This struct:

• Must implement the ockam::Worker trait.

• Must have the #[ockam::worker] attribute on the Worker trait implementation

• Must define two associated types Context and Message

For a new Echoer worker, create a new file at src/echoer.rs in your project. We're creating this inside the src directory so we can easily reuse the Echoer in other examples that we'll write later in this guide:

Add the following code to this file:

Note that we define the Message associated type of the worker as String, which specifies that this worker expects to handle String messages. We then go on to define a handle_message(..) function that will be called whenever a new message arrives for this worker.

In the Echoer's handle_message(..), we print any incoming message, along with the address of the Echoer. We then take the body of the incoming message and echo it back on its return route (more about routes soon).

To make this Echoer type accessible to our main program, export it from src/lib.rs file by adding the following to it:

App worker

When a new node starts and calls an async main function, it turns that function into a worker with address of "app". This makes it easy to send and receive messages from the main function (i.e the "app" worker).

In the code below, we start a new Echoer worker at address "echoer", send this "echoer" a message "Hello Ockam!" and then wait to receive a String reply back from the "echoer".

Create a new file at:

Add the following code to this file:

To run this new node program:

You'll see console output that shows "Hello Ockam!" received by the "echoer" and then an echo of it received by the "app".

Message Flow

The message flow looked like this:

Next, let’s explore how Ockam’s enables us to create protocols that provide end-to-end security and privacy guarantees.

Identities

Ockam Identities are cryptographically verifiable digital identities. Each Identity maintains one or more secret keys and has a unique Ockam Identifier.

When an Ockam Identity is first created, it generates a random primary secret key inside an Ockam Vault. This secret key must be capable of performing a ChangeSignature. We support two types of change signatures - EdDSACurve25519Signature or ECDSASHA256CurveP256Signature

This guide contains instructions to launch within AWS environment, an

• An Ockam Kafka Outlet Node within an AWS environment

• An Ockam Kafka Inlet Node:

  • Within an AWS environment, or

Transcript

Matthew Gregory: Welcome to the Ockam podcast. We have an exciting topic today, we are going to talk about a really cool use case for Ockam. This use case focuses on a setup of a company that's running a SaaS product in a cloud environment and needs to connect to their customer's data. We’ll discuss all the challenges that go into this problem of moving data from a customer to a SaaS product and what you need to think about. From the point of view of a product manager who's running the SaaS product, we’ll discuss the technical hurdles, things that happen to slow down sales, and ultimately what creates great customer success. That's the first thing that every product manager is thinking about. Given that all three of us are product managers at heart, I think we'll have a lot of good perspectives for the product managers listening. If you're on the technical implementation side, this discussion will run through several things that you may not have thought of in how to connect SaaS products to your customer's data. Let me start there, Glenn. What are some scenarios where a SaaS product would need to access customer data?

Why SaaS products need to connect to private systems