diff --git a/source/reference-manual/ota/production-targets.rst b/source/reference-manual/ota/production-targets.rst index c66b5c153..66197630c 100644 --- a/source/reference-manual/ota/production-targets.rst +++ b/source/reference-manual/ota/production-targets.rst @@ -35,6 +35,12 @@ Waves allow Factory operators to control an exact time when devices see a new ve Performing a Production OTA --------------------------- +.. important:: + + Before performing a Production OTA, ensure you have the `targets.key.tgz` in your + possession. This key is crucial for the operation and should be provided by the Factory admin. + For a deeper understanding of how this key is generated, refer to :ref:`ref-offline-keys`. + A user should define a process to select CI builds which need to be delivered to production devices. Let's assume a user selected a CI build version 42 as ready to be run in production. To start the production release process, a user would create a new wave using the below command:: @@ -45,9 +51,22 @@ To start the production release process, a user would create a new wave using th This creates a new TUF targets role version for production devices which listen to OTA updates for the ``production`` tag. That TUF targets role only includes a single Target from CI build (in above example, that target version is 42). -.. note:: +.. important:: We recommend that a user generates :ref:`OSTree static deltas` before rolling out waves to devices. + Static Deltas will optimize your OTA update download. + +.. note:: + + If you encounter issues while creating the weave, particularly the following error: + + .. code-block:: shell + + 409 CONFLICT + = Conflict: /ota/repo/qemu/api/v1/user_repo/root/updates? + = Only one TUF root updates transaction can be active at a time + + This indicates a conflict with an active TUF root update transaction. In such cases, it is advisable to contact the Factory admin. They should cancel the conflicting update by executing the `fioctl keys tuf updates cancel` command and then generate a fresh `targets.key.tgz` for your use. Once created, a new wave can be rolled out to Factory production devices, all at once or in phases. There are several ways how a wave can be rolled out: diff --git a/source/reference-manual/security/offline-keys.rst b/source/reference-manual/security/offline-keys.rst index f0d80e941..6f28e8db6 100644 --- a/source/reference-manual/security/offline-keys.rst +++ b/source/reference-manual/security/offline-keys.rst @@ -19,22 +19,163 @@ makes it hard to perform known attacks by obtaining only one of the secrets. FoundriesFactory provides the ability to manage TUF keys via dedicated `Fioctl™ `_ commands. +TUF Keys And Their Importance +-------------------------------- + +The Update Framework (TUF) provides a method to secure software update systems. +Central to TUF's security model are keys, which are cryptographic tools used to sign +and verify the authenticity of updates. In the context of the Factory, there are two +key types to be familiar with: + +**Root Key:** +Acts as the foundation of trust. This key signs metadata that declares which keys are +valid for other roles in the system. If compromised, an attacker could potentially +replace all other keys in the system. + +**Targets Key:** +Used to sign metadata that lists actual updates (or "targets"). A compromised targets +key could allow an attacker to issue malicious updates. + +TUF Roles And Their Importance +---------------------------------- + +The Update Framework (TUF) employs a hierarchy of roles to maintain the integrity and +security of software updates: + +**Root Role:** + +- Establishes trust for all other roles. +- Contains keys for other top-level roles. +- The foundation upon which trust for the entire system is built. + +**Timestamp Role:** + +- Ensures repository freshness. +- Points to the most recent snapshot. + +**Snapshot Role:** + +- Lists the latest versions of all update metadata. + +**Targets Role:** + +- Specifies available updates. +- Can delegate specific responsibilities to ensure granular control. + +Each role has its unique purpose, and by keeping these roles separate, +the system ensures that the compromise of one role doesn't jeopardize the entire +update process. + +How It All Comes Together +---------------------------------- + +Think of TUF as a secure delivery service: + +- The **Root** is the trust anchor, like the headquarters of this service. +- The **Timestamp** is like a date stamp ensuring the package (update) is fresh. +- The **Snapshot** provides a summary of the packages in the warehouse. +- And the **Targets** are the details about each package, ensuring you get the right item securely. + +The :ref:`Device Gateway ` checks each step, ensuring it gets a genuine, untampered update. +It's akin to a recipient verifying the authenticity and integrity of a parcel at every handover point, from the +warehouse to their doorstep. + +Online VS Offline Keys +----------------------- + +**Online Keys:** + +These are the keys that are stored on the servers and are accessible via the internet. +They are more prone to being compromised because they are connected to the internet and potentially susceptible +to a myriad of online attacks. + +**Offline Keys:** + +These keys are not connected to the internet. They are stored in a location that's offline. +Being offline makes them much less susceptible to online attacks, making them more secure. + +Why Rotate The TUF Root Key? +---------------------------- + +The TUF root key is the foundation of trust in the TUF system. It's essential to ensure that this key is as secure +as possible. By taking the key offline, you're protecting it from a wide range of potential online attacks. + +Rotating the key is the act of creating a new key and making it the active one. The rotation process involves several +steps to ensure a seamless transition and maintaining the integrity and trust of the TUF system. + +Why Rotate Regularly? +----------------------- + +Even when stored offline, keys can potentially be compromised. For instance, someone might gain physical +access to where they are stored, or there might be undetected vulnerabilities. Regular rotation ensures that even +if a key is compromised, its window of utility for malicious actors is limited. + +Summary Of Steps: +----------------------- + +Following a summary of the practical steps as an overview. + +**Initial Setup:** + +- The TUF root key is initially created online by Foundries.io™. +- The Factory admin using the following command should take the TUF roots key offline (`How to Rotate Offline TUF Root Key`_). **This is done as a one-time operation:** + +.. code-block:: shell + + fioctl keys tuf rotate-offline-key --role=root --first-time \ + --keys=/absolute/path/to/root.keys.tgz + +- The Factory admin must keep stored safe the ``root.keys.tgz`` of the keys which is created during this process. This backup (`Backup Offline TUF Keys`_) is vital for future operations and must be securely stored. Otherwise, it will no longer possible provide updates for the devices. + +**Target Key Generation and Management:** + +- After taking the TUF root key offline, the next important step is to manage the target keys which ensure the legitimacy of the updates. Use the following command to take the TUF targets key offline: + +.. code-block:: shell + + fioctl keys tuf rotate-offline-key --role=targets \ + --keys=/absolute/path/to/root.keys.tgz --targets-keys=/absolute/path/to/targets.keys.tgz + +- This command will generate the offline target keys (``targets.keys.tgz``) essential for signing the ``targets.json`` file. This ensures the authenticity and integrity of the updates. The ``targets.keys.tgz`` file, which results from this process, plays a crucial role when initiating weaves to advance Over-The-Air (OTA) updates for your devices. For instance, to create a weave, you must have the ``targets.keys.tgz`` to be used as for example in the following command: + +.. code-block:: shell + + fioctl wave init -k ~/path/to/keys/targets.only.key.tgz \ + update-prod-02 \ # Name of your wave update + 42 \ # The target ID that you want to promote + prod \ # The device tag which represent the devices that should receive the OTA update + +**Regular Rotations:** + +- After the initial setup, the TUF root key should be rotated regularly. **(at the least yearly)** +- The same command without the flag `--first-time` should be used for these regular rotations. +- Again, a backup of the keys (`Backup Offline TUF Keys`_) should be created and stored securely after each rotation. + .. note:: - We recommend running the command ``fioctl keys tuf updates review`` from time to time. - It will show you a list of our recommendations about improving your TUF usage, which evolve over time. - For example, it can show you the up to date information about the `Recommended Offline TUF Keys Schema`_. + In The Update Framework (TUF), the root key is the most critical, as it's the foundation for the trust chain. + The targets key, on the other hand, is used to sign the ``targets.json`` which represents actual software updates. -How to Rotate Offline TUF Root Key + It's wise to rotate the targets key regularly, but perhaps not as frequently as the root key. + The logic here is that while a compromised targets key could allow malicious updates, those updates would only be + considered valid for as long as that key is valid. Once you notice a malicious update, you would cancel the rollout + weave(s) which are using them, rotate the targets key to invalidate it and create new weave(s) with the new + Target keys to provide OTA updates. Depending on the threat model, some organizations might rotate the targets key + with every update, while others might do it on a set schedule (e.g., quarterly, bi-annually). + +Now, check the further sections for a deeper understand of those operations. + +How To Rotate Offline TUF Root Key ---------------------------------- The TUF root key is the most important key in TUF. The owner of this key can sign TUF ``root.json`` which defines what keys and roles your Factory devices can trust. -The initial TUF root key for your Factory is generated by Foundries.io™, and it is stored online on our servers. -The Factory admin must take that key offline by rotating it as soon as possible. -Thereafter, that (offline) TUF root key must be rotated regularly (at the least yearly) to keep your Factory secure. +The initial TUF root key for your Factory is generated by Foundries.io™ as described above, and it is stored online on our servers. +**The Factory admin must take that key offline by rotating it as soon as possible.** +Thereafter, that (offline) TUF root key must be rotated regularly **(at the least yearly)** to keep your Factory secure. .. note:: + In the past, we used to send the TUF keys tarball via email to Factory owners at Factory creation. That email can be ignored. Factory owners who received it and have not yet downloaded/rotated the initial keys, @@ -60,13 +201,16 @@ Onwards, use a shorter command to rotate your (offline) TUF root key:: .. note:: Here and below the ``root.keys.tgz`` file is the one created during the first offline TUF root key rotation. -When rotating the TUF root key, the newly generated key is added to the keys tarball (``root.keys.tgz`` in examples). -That file **must never be lost**. -Otherwise, it will be impossible to make any future updates to the Factory TUF keys. -That will lead to the inability to deliver new :ref:`Over-the-Air (OTA) updates ` to your Factory devices. -Therefore, after each TUF root key rotation, we recommend that you `Backup Offline TUF Keys`_ as described below. +.. important:: + You must create a backup of ``root.keys.tgz`` -How to Rotate Offline TUF Targets Key + When rotating the TUF root key, the newly generated key is added to the keys tarball (``root.keys.tgz`` in examples). + That file **must never be lost**. + Otherwise, it will be impossible to make any future updates to the Factory TUF keys. + That will lead to the inability to deliver new :ref:`Over-the-Air (OTA) updates ` to your Factory devices. + Therefore, after each TUF root key rotation, we recommend that you `Backup Offline TUF Keys`_ as described below. + +How To Rotate Offline TUF Targets Key ------------------------------------- TUF has the notion of a ``targets.json`` file which specifies what updates (Targets) are available to Factory devices. @@ -74,7 +218,7 @@ That file is automatically maintained by your Factory CI builds. At the end of each build it is updated, signed by (online) TUF targets key, and pushed to Foundries.io servers. Before going to production and creating the first :ref:`Wave and Production Targets `, -a Factory owner must generate the offline TUF targets key. +a Factory owner must generate the ``offline TUF targets key``. FoundriesFactory ensures that the Factory admins have exclusive control of updates to production devices. To achieve this, it requires production ``targets.json`` files to be signed by two TUF targets keys: @@ -84,6 +228,8 @@ To achieve this, it requires production ``targets.json`` files to be signed by t To generate your first offline TUF targets key, or rotate an existing key, use the following Fioctl_ command:: +.. code-block:: shell + fioctl keys tuf rotate-offline-key --role=targets \ --keys=/absolute/path/to/root.keys.tgz --targets-keys=/absolute/path/to/targets.keys.tgz @@ -95,6 +241,7 @@ This will perform the following steps: - and finally upload both new ``root.json`` and new production targets signatures to Foundries.io servers. .. note:: + Technically, that command generates two almost identical ``root.json`` files: the CI and production. The only difference is that the production ``root.json`` sets the targets signing threshold to 2, while the CI flavor of ``root.json`` keeps it at 1. @@ -102,17 +249,37 @@ This will perform the following steps: This is convenient to quickly test your CI builds on test devices, while keeping production devices more secured. .. note:: + The ``--targets-keys`` argument in the above command is optional; if it is missing, the command saves the offline targets key in the same file as the offline root key. We recommend keeping these keys in two separate files. That approach makes it possible to distribute the targets key among a wider set of Factory admins, and allow them to sign production targets without exposing the TUF root key to the wider audience. -After each TUF targets key rotation we recommend that you `Backup Offline TUF Keys`_ as described below. -If you lose the offline TUF targets key, a new key can be generated if you have your Factory offline TUF root key. -However, losing this key may be inconvenient if more than one Factory admin can manage production targets. +.. important:: + + After each TUF targets key rotation we recommend that you `Backup Offline TUF Keys`_ as described below. + If you lose the offline TUF targets key, a new key can be generated if you have your Factory offline TUF root key. + However, losing this key may be inconvenient if more than one Factory admin can manage production targets. + +The Need To View Offline TUF Keys +--------------------------------- + +Understanding when and why to view Offline TUF Keys is crucial for maintaining the security and trustworthiness of your update system. Here's a breakdown: -How to View Offline TUF Keys +**Why is it Important?** + +- Verification: Validating the integrity of updates requires making sure that the right keys are operational. +- Security Audit: Regularly inspecting the TUF keys ensures the Factory operates with optimal security and reduces risks of potential breaches. +- Troubleshooting: Should update-related issues arise, confirming the authenticity of keys becomes part of the resolution process. + +**When Should You Check?** + +- After Key Rotation: Once keys are rotated, it's pivotal to ascertain the new set is active and operational. +- Amidst Security Alerts: If there's any hint or evidence of a key compromise, you must conduct an immediate check to verify its authenticity. +- Routine Checks: Just like regular health check-ups, periodically inspect the keys as part of system maintenance and security hygiene. + +How To View Offline TUF Keys ---------------------------- The Factory's TUF metadata can be viewed using this Fioctl_ command:: @@ -163,7 +330,7 @@ See the section `How to Backup Offline TUF Keys`_ below, how the internal struct .. _Backup Offline TUF Keys: -How to Backup Offline TUF Keys +How To Backup Offline TUF Keys ------------------------------ There are 3 recommended ways for backing up your Factory TUF keys: @@ -179,22 +346,41 @@ See the section `How to View Offline TUF Keys`_ to understand the keys tarball i Expert Mode ----------- -The Fioctl_ ``keys tuf updates`` command set allows you to implement various TUF key workflows based on your requirements: +The ``fioctl keys tuf updates`` command set offers an advanced mode that enables administrators and experts to customize +and fine-tune their TUF key management. These specialized commands grant more control over the TUF key workflows, +enabling more secure and tailored configurations. + +Following a summary of the Expert Mode Features with Fioctl: -- Generate the TUF root versus the TUF targets keys on separate machines by different people. -- Make granular changes to your Factory TUF keys, e.g. rotate select online keys on Foundries.io servers. -- Add more than 1 offline TUF signing key for the TUF root or the production TUF targets role. -- Set a signature threshold for the TUF root or the production TUF targets role. +- **Transactional Staging:** -Please, run the command ``fioctl keys tuf updates --help`` to view the examples. + - **Initiate Transaction**: Begin a TUF updates transaction with ``fioctl keys tuf updates init``. This provides a unique Transaction ID for other associated actions. + - **Review Changes**: Inspect staged changes using ``fioctl keys tuf updates review``. + - **Apply or Cancel**: Depending on the review, either apply the changes with ``fioctl keys tuf updates apply`` or cancel with ``fioctl keys tuf updates cancel``. -One command of interest is ``fioctl keys tuf updates review``. -It analyses your Factory's existing TUF root on the server, and prints a list of recommendations for improving it. -Over time our engineers will add more items to that list as we develop new security features. +- **Key Management:** + + - **Take Ownership**: For a new factory, own the TUF root and targets keys, possibly on separate machines. + - **Add Offline Key**: Introduce an offline TUF signing key using ``fioctl keys tuf updates add-offline-key``. + - **Rotate Keys**: Allow rotation for both offline with ``fioctl keys tuf updates rotate-offline-key`` and online TUF signing keys with ``fioctl keys tuf updates rotate-online-key``. + - **Delete Offline Key**: Remove an offline TUF signing key using ``fioctl keys tuf updates delete-offline-key``. + +- **Signature Management:** + + - **Set Signature Threshold**: Define the necessary number of signatures for TUF root or production targets using ``fioctl keys tuf updates set-threshold``. + - **Offline Signing**: Sign the staged TUF root using `fioctl keys tuf updates sign`` and sign production targets with ``fioctl keys tuf updates sign-prod-targets``. + +For a comprehensive list of commands and their usage examples, refer to the ``fioctl keys tuf updates --help`` +command output or consult the documentation. + +.. note:: + We recommend running the command ``fioctl keys tuf updates review`` from time to time. + It will show you a list of our recommendations about improving your TUF usage, which evolve over time. + For example, it can show you the up to date information about the `Recommended Offline TUF Keys Schema`_. .. _ref-offline-keys-more-than-1-root: -How to Add More Than 1 Offline TUF Keys +How To Add More Than 1 Offline TUF Keys +++++++++++++++++++++++++++++++++++++++ Usually, you need to add more than 1 offline TUF signing key for your TUF roles in one of these use cases: @@ -265,7 +451,7 @@ At any moment before applying the changes, and admin can cancel the transaction Any user with admin rights can cancel the TUF root updates transaction, not only the one who initiated it. -How to Increase the TUF Signature Threshold +How To Increase the TUF Signature Threshold +++++++++++++++++++++++++++++++++++++++++++ Requiring more than 1 offline signature for any TUF root changes greatly improves the TUF root role security.