Skip to content

Write an enclave host

Jump to bottom
JohnJoser3 edited this page Jan 11, 2023 · 2 revisions

Writing your own enclave host

Prerequisites

Complete the following tutorials:

Introduction

Conclave projects consist of three modules: the client, the host, and the enclave.

The host is responsible for:

  1. Instantiating the enclave.
  2. Persisting data to disk.
  3. Passing messages between the enclave and its clients.

Conclave web host is a built-in web host that manages these tasks for simple use cases. You can implement a custom host for complex use cases.

Project setup

To implement a simple host server using raw sockets:

  1. Create a new Conclave project using Conclave Init and implement your enclave.
  2. Create a main class for the new host.
package com.example.tutorial.host;

public class MyEnclaveHost {
    public static void main(String[] args) {

    }
}
  1. Update the build.gradle file to reference the main class:
application {
    mainClass.set("com.example.tutorial.host.MyEnclaveHost")
}
  1. Replace the runtimeOnly conclave-web-host dependency with implementation conclave-host:
dependencies {
    runtimeOnly project(path: ":enclave", configuration: mode)
    implementation "com.r3.conclave:conclave-host:$conclaveVersion"
}
  1. Remove the generated client code and create a blank main class:
package com.example.tutorial.client;

class MyEnclaveClient {
    public static void main(String[] args) {

    }
}
  1. Replace the conclave-web-client dependency with conclave-client:
dependencies {
    implementation "com.r3.conclave:conclave-client:$conclaveVersion"
}
  1. Check that the host and client run without any issues:
./gradlew :host:run
./gradlew :client:run

Implementing the client

Use the EnclaveClient class to manage communication with the enclave. It deals with the encryption of Conclave Mail messages and simplifies enclave restarts.

You can use the EnclaveTransport interface to handle the details of the transport layer to the host. The EnclaveClient.start method needs an implementation of the EnclaveTransport class. For example, if the enclave is running behind the Conclave web host, then the client needs to use the WebEnclaveTransport class.

This sample uses a simple, socket-based EnclaveTransport. It requires host and port parameters.

public class MyEnclaveTransport implements EnclaveTransport, Closeable {
    private final String host;
    private final int port;
    private Socket socket;
    private DataInputStream input;
    private DataOutputStream output;

    public SocketEnclaveTransport(String host, int port) {
        this.host = host;
        this.port = port;
    }

    public void start() throws IOException {
        socket = new Socket(host, port);
        input = new DataInputStream(socket.getInputStream());
        output = new DataOutputStream(socket.getOutputStream());
    }

    @Override
    public void close() throws IOException {
        if (socket != null) {
            socket.close();
        }
    }
}

This sample implements Closeable on the host and port parameters of the SocketEnclaveTransport() method to allow the caller to close any underlying connections. The input and output streams are the communication channels with the host server for receiving and sending raw bytes.

enclaveInstanceInfo()

The first EnclaveTransport method you need to implement is enclaveInstanceInfo, which downloads the latest EnclaveInstanceInfo object from the host:

@NotNull
@Override
public EnclaveInstanceInfo enclaveInstanceInfo() throws IOException {
    output.write(1);
    output.flush();

    byte[] attestationBytes = new byte[input.readInt()];
    input.readFully(attestationBytes);
    return EnclaveInstanceInfo.deserialize(attestationBytes);
}

The attestation request is a single-byte value. When the client sends the byte, the enclaveInstanceInfo() method waits for the server to respond with the attestation bytes. When the method receives the attestation bytes from the server, it deserializes the attestation bytes into an EnclaveInstanceInfo object.

connect()

Next, you need to implement the methods to send and receive Conclave Mail. The ClientConnection interface defines these methods. You can create an instance of [ClientConnection] using EnclaveTransport.connect. Multiple EnclaveClient instances can use a single EnclaveTransport interface. The ClientConnection implementation is a private inner class and connect will simply return a new instance of one.

@NotNull
@Override
public ClientConnection connect(@NotNull EnclaveClient client) throws IOException {
    return new MyClientConnection();
}

private class MyClientConnection implements ClientConnection {
    @Override
    public void disconnect() {
    }
}

The disconnect method is empty in this simple implementation as there's nothing to do when the EnclaveClient closes.

sendMail()

To send encrypted Mail to the host, you need to implement sendMail:

@Nullable
@Override
public byte[] sendMail(@NotNull byte[] encryptedMailBytes) throws IOException, MailDecryptionException {
    output.write(2);
    output.writeInt(encryptedMailBytes.length);
    output.write(encryptedMailBytes);
    output.flush();

    int responseType = input.readByte();
    if (responseType == 1) {
        return readMail();
    } else if (responseType == 2) {
        throw new MailDecryptionException();
    } else {
        throw new IOException("Unknown response type " + responseType);
    }
}

The sendMail() request has a byte value 2, followed by the size prefix. After sending this request, the client waits for a response from the host. The sendMail specification states that the method must block and wait for the enclave to process the Mail. If the enclave processes the Mail successfully, the client must receive and return any response from the enclave. A response type 1 represents such a success response from the client.

private byte[] readMail() throws IOException {
    int responseMailSize = input.readInt();
    if (responseMailSize > 0) {
        byte[] responseMail = new byte[responseMailSize];
        input.readFully(responseMail);
        return responseMail;
    } else {
        return null;
    }
}

If the enclave couldn't decrypt the Mail, the client must throw a MailDecryptionException. A response type 2 represents such a response from the client.

pollMail()

The final method you need to implement is pollMail which is for polling for any extra response Mail the enclave might have created for the client.

@Nullable
@Override
public byte[] pollMail() throws IOException {
    output.write(3);
    output.flush();

    return readMail();
}

A single-byte prefix represents the pollMail() request. You don't need to send any other parameters. The response follows the same path as sendMail if it receives a Mail response. So you can reuse the readMail() method from above.

You can implement EnclaveTransport with an EnclaveClient instance to connect to the host.

public static void main(String[] args) throws InvalidEnclaveException, IOException {
    EnclaveClient client = new EnclaveClient(EnclaveConstraint.parse(args[0]));
    MyEnclaveTransport enclaveTransport = new MyEnclaveTransport("localhost", 8000);
    enclaveTransport.start();
    client.start(enclaveTransport);
    // Send and receive mail
}

You need to implement the corresponding logic on the host to receive and process the requests.

Implementing the host

Loading the enclave

Load the enclave using EnclaveHost.load().

public class MyEnclaveHost {
    private static EnclaveHost enclaveHost;
    
    public static void main(String[] args) throws EnclaveLoadException, IOException {
        enclaveHost = EnclaveHost.load();
    }
}

!!!Note

In projects containing multiple enclave modules, you can specify the enclave to load by using the fully qualified
class name:
```java
enclaveHost = EnclaveHost.load("com.example.tutorial.enclave.MyEnclave");
```

Starting the enclave

Start the enclave by calling EnclaveHost.start.

Path hostDir = Paths.get(args[0]);
String kdsUrl = args[1];

enclaveStateFile = hostDir.resolve("enclave.state");
Path enclaveFileSystemFile = hostDir.resolve("enclave.fs");

byte[] sealedState;
if (Files.exists(enclaveStateFile)) {
    sealedState = Files.readAllBytes(enclaveStateFile);
} else {
    sealedState = null;
}

In this tutorial, you can pass the parameters of EnclaveHost.start method from the command line. The first parameter is a reference to the host directory, which will contain the file for the enclave's encrypted file system and a file for the enclave's sealed state. This parameter is optional.

Note

The enclave's sealed state should ideally be stored in a database and committed as part of the same transaction that processes outbound Mail from the enclave. This is why the sealed state parameter is a byte array and not a file path. You can find more information about this here.

The second command line parameter is for the URL to the Key Derivation Service (KDS). You can leave out this parameter if your enclave doesn't use the KDS.

Call enclaveHost.start:

enclaveHost.start(
        new AttestationParameters.DCAP(),
        sealedState,
        enclaveFileSystemFile,
        new KDSConfiguration(kdsUrl),
        (commands) -> {
            try {
                processMailCommands(commands);
            } catch (IOException e) {
                throw new UncheckedIOException(e);
            }
        }
);

System.out.println(enclaveHost.getEnclaveInstanceInfo());

The last start parameter is a callback lambda for processing Mail commands from the enclave. The commands come from the enclave grouped together in a list after every callEnclave or deliverMail call.

When the enclave starts, the host logs the enclave's attestation report to the console. You can use these logs for debugging and choosing the enclave constraint when running the client.

You can find a detailed explanation of the start parameters in the API docs.

Accepting the client connection

Now the enclave is ready to receive Mail. For this, you need to set the host to listen on a port for a client to connect to. You can do this by passing a server port from the command line:

int serverPort = Integer.parseInt(args[2]);
ServerSocket serverSocket = new ServerSocket(serverPort);
System.out.println("Listening on port " + serverPort);

Implementing the request loop

Next, set up the request loop:

Socket clientSocket = serverSocket.accept();
System.out.println("Client connected");

DataInputStream input = new DataInputStream(clientSocket.getInputStream());
DataOutputStream output = new DataOutputStream(clientSocket.getOutputStream());

while (true) {
    int requestType = input.read();
    if (requestType == -1) {
        System.out.println("Client disconnected");
        break;
    }
    if (requestType == 1) {
        sendAttestation(output);
    } else if (requestType == 2) {
        processInboundMail(input, output);
    } else if (requestType == 3) {
        sendPostedMail(output);
    } else {
        System.err.println("Unknown request type " + requestType);
    }
}

serverSocket.close();

!!!Note

This host implementation accepts only a single client connection. To support multiple concurrent clients, you need
to make necessary changes to both the host and the client.

The input and output objects receive and send bytes to the client, respectively. The first thing to do is to block and wait for the first byte from the client. You can implement the different methods available depending on the request type as given below:

Attestation request

The attestation request is straightforward to implement as it's just sending the serialized EnclaveInstanceInfo:

private static void sendAttestation(DataOutputStream output) throws IOException {
    byte[] attestationBytes = enclaveHost.getEnclaveInstanceInfo().serialize();
    output.writeInt(attestationBytes.length);
    output.write(attestationBytes);
    output.flush();
}

Mail request

The next request to implement is sendMail:

private static void processInboundMail(DataInputStream input, DataOutputStream output) throws IOException {
    byte[] mailBytes = new byte[input.readInt()];
    input.readFully(mailBytes);
    try {
        enclaveHost.deliverMail(mailBytes, null);
        sendPostedMail(output);
    } catch (MailDecryptionException e) {
        output.write(2);
        output.flush();
    }
}

When the host receives the Mail bytes, the host uses the deliverMail method to deliver the Mail bytes to the enclave.

The enclave needs a routing hint parameter to route responses back to clients if there are multiple clients. This implementation doesn't use a routing hint, as only one client exists.

The deliverMail method throws a MailDecryptionException if the enclave cannot decrypt the Mail bytes. You need to notify this exception so that the client can send back a response value of 2, which the earlier implementation of sendMail expects.

Mail commands

Now you need to implement the Mail commands introduced earlier where the call to EnclaveHost.start references a processMailCommands method:

private static final Queue<byte[]> postedMail = new LinkedList<>();

private static void processMailCommands(List<MailCommand> commands) throws IOException {
    for (MailCommand command : commands) {
        if (command instanceof MailCommand.PostMail) {
            MailCommand.PostMail postMail = (MailCommand.PostMail) command;
            postedMail.add(postMail.getEncryptedBytes());
        } else if (command instanceof MailCommand.StoreSealedState) {
            MailCommand.StoreSealedState storeSealedState = (MailCommand.StoreSealedState) command;
            Files.write(enclaveStateFile, storeSealedState.getSealedState());
        }
    }
}

Mail responses from the enclave go through the PostMail command. In this tutorial, you store the Mail responses in a queue.

This tutorial also uses the StoreSealedState command. This command overwrites the disk with the new sealed state.

You need to implement sendPostedMail, which takes the first response Mail from the queue, and sends it to the client:

private static void sendPostedMail(DataOutputStream output) throws IOException {
    byte[] mailResponse = postedMail.poll();
    output.write(1);
    if (mailResponse != null) {
        output.writeInt(mailResponse.length);
        output.write(mailResponse);
    } else {
        output.writeInt(0);
    }
    output.flush();
}

As the pollMail request also uses the same logic, the request loop above calls sendPostedMail if it receives a request type 3.

Clone this wiki locally