Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: improve the document that describes the portals app #7216

Merged
merged 1 commit into from
Dec 22, 2023
Merged

docs: improve the document that describes the portals app #7216

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 37 additions & 31 deletions examples/app/portals/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,62 +1,68 @@
# Encrypted Portals between Macs – built in Swift and Rust

- [What is a Portal and how you can use it.](#what-is-a-portal-and-how-you-can-use-it)
- [How we built a Swift macOS app that uses our Rust library.](#how-we-built-a-swift-macos-app-that-uses-our-rust-library)
- [The protocols that enable remote TCP service to appear on localhost next to a client.](#the-protocols-that-enable-remote-tcp-service-to-appear-on-localhost-next-to-a-client)
- [Portals for Mac](#portals-for-mac)
- [Introduction to Ockam Portals](#introduction-to-ockam-portals)
- [How we built a Swift macOS app that uses our Rust library](#how-we-built-a-swift-macos-app-that-uses-our-rust-library)

## What is a Portal and how you can use it
## Portals for Mac

A TCP Portal makes a remote TCP service appear on localhost, virtually adjacent to TCP clients.
Portals is a Mac app built in Swift. It uses the Ockam Rust library to privately share a service on your Mac to anyone, anywhere. The service is shared securely over an end-to-end encrypted Ockam Portal. Your friends will have access to it on their **localhost**!

All communication happens over end-to-end encrypted and mutually authenticated Ockam Secure Channels. Channels are established over multi-hop, multi-protocol transport routes that can include bridges, relays, or rendezvous. This enables end-to-end encrypted portals that can traverse NATs, firewalls, and clouds without any change to networks or infrastructure.
<img width="1012" src="https://github.com/build-trust/ockam/assets/159583/5efbcbfa-5083-4682-941c-71d9a6c24f84">

TCP Portals are different from VPNs because there is no virtual IP network, there is only a single virtualized point-to-point TCP connection over an end-to-end encrypted channel. TCP Portals are also different from reverse proxies and load balancers because there is no exposed entrypoint to the Internet. The two ends of an Ockam Portal can live in completely private networks that don't expose any listening ports or allow any ingress. Unlike TLS termination at loadbalancers, end-to-end Ockam Secure Channels do not expose your application's data to a third party operator. Data authenticity, integrity, and confidentiality are guaranteed between the two ends.
Let's see Portals in action in this quick 2 minute video (please unmute for an explanation of what is happening):

Under the covers Ockam Secure Channels use lightweight and robust cryptographic primitives that have been proven, at scale, within modern systems like Signal, Whatsapp, and Wiregaurd. The design of our open source stack of [protocols](#the-protocols-that-enable-remote-tcp-service-to-appear-on-localhost-next-to-a-client) and their use for TCP Portals was recently audited by the security research firm Trail of Bits. The executive summary of their report states: _"None of the identified issues pose an immediate risk to the confidentiality and integrity of data handled by the system in the context of the two in-scope use cases (TCP Portals and Kafka Portals). The majority of identified issues relate to information that should be added to the design documentation."_ We're very excited about the report and in the coming weeks, we'll share this report and a detailed writeup about the audit process.
https://github.com/build-trust/ockam/assets/159583/6e883e57-65c3-46d2-a05a-41fab4299c71

You can create production ready portals using Ockam Command to privately connect applications across companies, VPCs, regions, clouds, and data centers. Our team is fully remote and we wanted an easy GUI to privately share services with our teammates and friends. Everyone on our team uses a Mac so we created an open source macOS menubar app using SwiftUI and the Ockam Rust library.
### Install

<img width="1012" src="https://github.com/build-trust/ockam/assets/159583/5efbcbfa-5083-4682-941c-71d9a6c24f84">
If you use [Homebrew](https://brew.sh/):

Each TCP Portal has two parts:
1. A TCP Outlet runs adjacent to a TCP server. The outlet decrypts and unwraps all Ockam Routing information and delivers raw TCP messages to the server. It also encrypts and wraps messages destined for clients with Ockam Routing information which allows these messages to be delivered to the corresponding Inlets.
2. A TCP Inlet that runs adjacent to one or more TCP clients. The inlet encrypts and wraps any messages destined for the server in Ockam Routing information which allows these messages to be delivered to the corresponding Outlets. The inlet also decrypts and unwraps all Ockam Routing information and delivers raw TCP messages to the clients.
```
brew update && brew install --cask build-trust/ockam/portals
```

If the outlet is within a private network, the outlet and inlet nodes only make outgoing TCP connections and the outlet is made reachable to inlets using an encrypted relay or a NAT puncturing rendezvous. An inlet and an outlet are mutually authenticated using unique cryptographic identities and credentials. Each connection is also authorized using granular attribute-based access control policies.
Or, install it using the appropriate dmg file for your Mac:
- Apple Silicon based Mac: Install from [this dmg file](https://github.com/build-trust/ockam/releases/download/ockam_v0.113.0/ockam.app.aarch64-apple-darwin.dmg).
- Intel based Mac: Install from [this dmg file](https://github.com/build-trust/ockam/releases/download/ockam_v0.113.0/ockam.app.x86_64-apple-darwin.dmg).

<img width="1012" src="https://github.com/build-trust/ockam/assets/159583/cb40efe5-001d-4c04-aba0-9530e163abf2">
## Introduction to Ockam Portals

### Portals.app
An Ockam Portal carries a non-Ockam protocol over Ockam. There are various types of Ockam Portals: TCP Portals, UDP Portals, Kafka Portals, etc. The Portals app for Mac supports TCP Portals.

First, let's install the app. If you use [Homebrew](https://brew.sh/), then you can install with this simple command:
Ockam Portals are built on top of [Ockam Routing](https://docs.ockam.io/reference/protocols/routing), and end-to-end encrypted Ockam [Secure Channels](https://docs.ockam.io/reference/protocols/secure-channels). Channels can be established over multi-hop, multi-protocol transport routes that may include bridges, relays, or rendezvous. This layering enables end-to-end encrypted Portals. It also allows Portals to traverse NATs, firewalls, and clouds without any change to networks or infrastructure.

```
brew update && brew install --cask build-trust/ockam/portals
```
Under the covers, Ockam Secure Channels use lightweight and robust cryptographic primitives that have been proven, at scale, within modern systems like Signal, Whatsapp, and Wiregaurd. The design of Ockam's open source stack of protocols, and their use for TCP Portals, was audited by the security research firm Trail of Bits. They identified no immediate risks to the confidentiality or integrity of application data moving through Ockam. [A link to the final report will appear here when it's released in January]

If you prefer to install the app manually, download and install it using the appropriate dmg file for your Mac. If you have an Apple Silicon based Mac, install from [this dmg file](https://github.com/build-trust/ockam/releases/download/ockam_v0.113.0/ockam.app.aarch64-apple-darwin.dmg). If you have an Intel based Mac, install from [this dmg file](https://github.com/build-trust/ockam/releases/download/ockam_v0.113.0/ockam.app.x86_64-apple-darwin.dmg).
A TCP Portal carries TCP over Ockam. It makes a remote TCP service appear on localhost; virtually adjacent to a local TCP client.

Next, let's see the Portals.app in action in this quick 2 minute video (please unmute for an explanation of what is happening):
<img width="1012" src="https://github.com/build-trust/ockam/assets/159583/44bdfae0-fdb7-404f-8e2d-f08981c19076">

https://github.com/build-trust/ockam/assets/159583/6e883e57-65c3-46d2-a05a-41fab4299c71
TCP Portals are different from VPNs because there is no virtual IP network. There is only a single, virtualized point-to-point TCP connection over an end-to-end encrypted channel. TCP Portals are also different from reverse proxies and load balancers because there is no exposed entrypoint on the Internet. The two ends of an Ockam Portal can live in completely private networks, that don't expose any listening ports, or allow any ingress at the network layer. Unlike TLS termination at loadbalancers, end-to-end Ockam Secure Channels do not expose your application's data to a third party operator. Data authenticity, integrity, and confidentiality are guaranteed between the two ends - your TCP service and its clients.

Each TCP Portal has two parts:
1. A TCP Outlet that runs adjacent to a TCP server. The Outlet decrypts and unwraps all Ockam Routing metadata and delivers raw TCP messages to the server. It also encrypts and wraps messages destined for clients with Ockam Routing metadata which allows these messages to be delivered to the corresponding Inlets.
2. A TCP Inlet that runs adjacent to one or more TCP clients. The Inlet encrypts and wraps any messages destined for the server in Ockam Routing metadata which allows these messages to be delivered to the corresponding Outlets. The Inlet also decrypts and unwraps all Ockam Routing metadata and delivers raw TCP messages to the clients.

If the Outlet is within a private network, the Outlet and Inlet nodes only make outgoing TCP connections, and the Outlet is made reachable to Inlets using an encrypted relay or a NAT puncturing rendezvous. An Inlet and an Outlet are mutually authenticated using unique cryptographic [identities and credentials](https://docs.ockam.io/reference/protocols/identities). Each connection is also authorized using granular [attribute-based access control](https://docs.ockam.io/reference/protocols/access-controls) policies.

<img width="1012" src="https://github.com/build-trust/ockam/assets/159583/cb40efe5-001d-4c04-aba0-9530e163abf2">

## How we built a Swift macOS app that uses our Rust library

The functionality, of the app, was already implemented in Ockam rust crates. So our focus was building a great native macOS experience.
The functionality of the Portals app was already implemented in the Ockam Rust library. All we had to do was create a great macOS-native experience.

Our first attempt at building the app was using Tauri. This made sense as we wanted to use the Ockam rust library and most people on our team are comfortable building things in Rust. This first version was easy to build and had all the basic functions we wanted. However, the experience of using the app wasn't great. Tauri only gave us minimal control over how the menu was rendered and what happened when a user interacts with the menu. This version of the app felt like it belonged in a 10 year old version of macOS when compared to super easy to use menubar items built into macOS Sonoma.

We realized that to have the rich experience we want we must build the app using SwiftUI.

Unfortunately, we couldn't find a an off-the-shelf solution to integrate Swift and Rust that would give us the best of both worlds - the safety of using rust and rich macOS-native experience of SwiftUI. Digging in a some more, the command ground is - Rust is compatible with the C calling convention and Swift is interoperable with Objective-C which is a superset of C-89 - so we can connect the two worlds using C-89.
We realized that to have the rich experience we want, we must build the app using SwiftUI.

Here's what that looks like:
Unfortunately, we couldn't find an off-the-shelf solution, to integrate Swift and Rust, that would give us the best of both worlds; the safety of Rust, and the rich macOS-native experience of SwiftUI. After some more digging we realized we can connect the two using C-89. Rust is compatible with the C calling convention, and Swift is interoperable with Objective-C, which is a superset of C-89.

<img width="1012" src="https://github.com/build-trust/ockam/assets/159583/b5e691bd-fd96-41f0-922a-d32d7bf12f34">

We wrote the Rust data structures that needed to be visible to Swift twice. One version is idiomatic in Rust and easy to use. The other version is C compatible using pointers and memory that is manually allocated with `malloc`. We also exposed some C-compatible APIs that use raw-pointers in unsafe rust to converted the idiomatic data structures to their C-compatible versions. Finally we automatically generated a C header with the help of the `cbindgen` library.
We wrote the Rust data structures that needed to be visible to Swift twice. One version is idiomatic in Rust and easy to use. The other version is C compatible using pointers and memory that is manually allocated with `malloc`. We also exposed some C-compatible APIs that use raw-pointers in unsafe rust to convert the idiomatic data structures to their C-compatible versions. Finally we automatically generated a C header with the help of the `cbindgen` library.

On the Swift side, we could've called the C APIs directly but C data structures are not first class citizens in Swift which makes them harder to use idiomatically within SwiftUI code. We instead chose to duplicate the data structures in Swift and convert between C and Swift. This may seem like a maintenance burden, but practically the state shared between the two worlds doesn't change that often while the ability to quickly build components in SwiftUI using constructs like `if let ..`, `ForEach`, `enum` etc. is super helpful.
On the Swift side, we could have called the C APIs directly, but C data structures are not first class citizens in Swift. This makes them harder to use idiomatically within SwiftUI code. Instead, we chose to duplicate the data structures in Swift and convert between C and Swift. This may seem burdensome, but practically, the shared state doesn't change very often. The ability to quickly build components in SwiftUI using constructs like `if let ..`, `ForEach`, `enum` etc. is super helpful and worth the tradeoff.

Here's an example of the same structure in its 4 forms:

Expand Down Expand Up @@ -100,7 +106,7 @@ class LocalService {
}
```

The Swift app is statically linked to our Rust lib at compile time. Data flow is simple. UI interactions are sent from Swift to Rust as actions by calling C APIs. Change events are emitted only by Rust, Swift is notified using callbacks that lead to updates to the user interface.
The Swift app is statically linked to our Rust lib at compile time. The data flow is simple: UI interactions are sent from Swift to Rust as actions by calling C APIs, change events are emitted only by Rust, and Swift is notified using callbacks that lead to updates to the UI.

Most code in the SwiftUI views looks just like any other SwiftUI application.

Expand Down
Loading