Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rm: linux-disk-encryption: document re-encryption #621

Closed
wants to merge 7 commits into from
Closed

rm: linux-disk-encryption: document re-encryption #621

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d375aed
rm: linux-disk-encryption: document re-encryption
ldts Nov 1, 2023
efb04d2
Update source/reference-manual/linux/linux-disk-encryption.rst
ldts Nov 3, 2023
79977a1
Update source/reference-manual/linux/linux-disk-encryption.rst
ldts Nov 3, 2023
38c5b50
Update source/reference-manual/linux/linux-disk-encryption.rst
ldts Nov 3, 2023
3a71ee8
Update source/reference-manual/linux/linux-disk-encryption.rst
ldts Nov 3, 2023
285b686
Update source/reference-manual/linux/linux-disk-encryption.rst
ldts Nov 3, 2023
aa10607
Merge branch 'main' into luks
kprosise Dec 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added source/_static/systemd-luks.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
341 changes: 328 additions & 13 deletions source/reference-manual/linux/linux-disk-encryption.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,80 @@
Disk Encryption Support
=======================

LmP supports encrypting the disk used by the root file system on first boot.
This is to guarantee a unique LUKS2 master key per device.
To enhance the security of system deployments before manufacturing
provisions the Targets, LmP takes the approach detailed here.

During the image creation process in CI, LmP uses LUKS to construct a
symmetrically encrypted root file-system. The master key used for
data encryption is locked/unlocked via a passphrase.

Upon the initial boot, during initramfs, the passphrase used for
accessing the master key is discarded and replaced by a cryptographic
key. Since this key is used to protect the master key, it is known as
the Key Encryption Key (KEK).

LmP uses LUKS which uses Systemd to register the new KEK. The KEK is an

Check warning on line 18 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 18, "column": 59}}}, "severity": "INFO"}
RSA 2048 asymmetric key, with the key pair being generated via either a
TPM 2.0 or a PKCS#11 security token. This also ensures the secure
storage of the key information.

By enrolling the KEK this way, the system gains the capability of
unlocking the volume seamlessly, without requiring any user involvement.

Once the KEK has been enrolled, LmP marks the rootfs volume for disk

Check warning on line 26 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 26, "column": 1}}}, "severity": "INFO"}
`re-encryption`_, mounts it and allows the system to continue with the
boot sequence. This is know in LUKS terminology as ``online
re-encryption``.

As part of the regular boot procedure, a systemd service will initiate

Check warning on line 31 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 31, "column": 1}}}, "severity": "INFO"}
the data re-encryption process in the background; this process is
non-blocking and allows the rest of the system to continue initializing.

Upon completion of the re-encryption, the service also creates and
stores a backup of the LUKS2 header. This backup image is stored in the

Check warning on line 36 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 36, "column": 38}}}, "severity": "INFO"}
primary partition and serves as a safeguard, enabling the booting of
systems in cases where their disk headers may have been damaged.

.. note::

If the system is restarted before the non-blocking re-encryption

Check warning on line 42 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 42, "column": 3}}}, "severity": "INFO"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ldts Are we 100% sure this is true? We have clients that are using re-encryption and it picks up where it left off and does not block the boot in their case. Did something change here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes 100%.
I am not sure what configuration they might be running but in our case as soon as we enroll the TPM/PKCS11 tokens, we remove the passphrase and initiate the re-encryption.

If the volume is closed without having completed re-encryption, it just cant be opened again. It is easy to prototype locally on any machine (create a file of ~60MB, create an ext4 filesystem, and encrypt it with luks).

but tell me more about the configuration of those clients so I can see how they differ...

Copy link
Contributor Author

@ldts ldts Nov 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

um, are you sure about what your clients are reporting??

see this:
https://github.com/foundriesio/meta-lmp/blob/main/meta-lmp-base/recipes-core/initrdscripts/initramfs-framework/cryptfs#L109

with the current code in the baseline, if online reencryption didnt finish and the board reboots we are going to block initramfs until it completes (resume is a blocking call)

having said that, this segment of code seems to succeed at opening ? which is kind of weird...

service has finished, the subsequent boot will be **blocked in
initramfs** until the root file system has been fully encrypted.

The effort for creating an encrypted root file system via CI (and required logic for online re-encryption during first boot) is part of ``meta-lmp`` `pull request 868 <https://github.com/foundriesio/meta-lmp/pull/868>`.

Prerequisites
-------------

As the process for decrypting the disk needs to be unattended, LmP requires either PKCS#11 (e.g. OP-TEE with RPMB as secure storage) or TPM 2.0 to be available by the target hardware.
These are leveraged for securely storing the Key Encryption Key (KEK) used for decrypting the disk during the boot process.
This is done via LUKS2 tokens, leveraging systemd-pkcs11 or systemd-tpm2, see `systemd-cryptenroll`_ for more information.
To ensure that the disk encryption and decryption processes can be

Check warning on line 50 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 50, "column": 1}}}, "severity": "INFO"}
carried out without human intervention, LmP mandates the presence of
either PKCS#11 or TPM 2.0 controlled devices on the target hardware.

Devices controlled via PKCS#11 or TPM 2.0 interfaces can therefore be
utilized to securely store and provide the Key Encryption Key (KEK).

Storing the KEK is achieved through the use of LUKS2 tokens, making use

Check warning on line 57 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 57, "column": 1}}}, "severity": "INFO"}
of either the **systemd-pkcs11** or **systemd-tpm2** plugins (refer to
`systemd-cryptenroll`_ for additional details).

For enhanced security, TPM 2.0 support also requires UEFI secure boot to be enabled.
This is because the key is bound to the Platform Configuration Register (PCR) 7, which tracks the secure boot state of the machine.

.. figure:: /_static/systemd-luks.png
:width: 300
:align: center

systemd-cryptenroll


In ARM System on Chips (SoCs), a common PKCS#11 scenario is to execute
OP-TEE within the TrustZone. In this setup, OP-TEE should be configured
to use the eMMC Replay Protected Memory Block (RPMB) for secure storage
with tamper-resistant properties.

In TPM 2.0 devices and as a security enhancement, we require that UEFI
boots with secure boot be enabled. This is because the KEK is linked to
the Platform Configuration Register 7 (PCR 7), which monitors the
**secure boot state of the machine**.

We demonstrate both of these scenarios using QEMU in the sections below.

Enabling Support for Disk Encryption
------------------------------------
Expand Down Expand Up @@ -69,7 +129,7 @@
A recovery key can be created and provided manually if required, but it will not be an unattended boot.

Testing TPM 2.0 Support With Qemu (x86) and swtpm
-------------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It is possible to test the disk encryption support with TPM 2.0 with QEMU and `swtpm`_.

Expand All @@ -80,8 +140,9 @@
$ cat meta-subscriber-overrides/conf/machine/include/lmp-factory-custom.inc
DISTRO_FEATURES:append:intel-corei7-64 = " luks"

Then enroll the :ref:`UEFI Secure Boot Certificates <ref-secure-boot-uefi>` to enable secure boot support.
This is required as the LUKS2 TPM 2.0 token leverages PCR 7, which tracks the secure boot state.
Then make sure to enroll the :ref:`UEFI Secure Boot Certificates <ref-secure-boot-uefi>`
to enable secure boot support. This is required as the LUKS2 TPM 2.0 token
leverages **PCR 7**, which tracks the secure boot state.

Now install ``swtpm`` on the host machine, and start the ``swtpm`` daemon.
This will be consumed by QEMU and act as the hardware TPM.
Expand Down Expand Up @@ -139,14 +200,14 @@
Label: otaroot
Subsystem: (no subsystem)
Flags: (no flags)

Data segments:
0: crypt
offset: 16777216 [bytes]
length: (whole device)
cipher: aes-xts-plain64
sector: 512 [bytes]

Keyslots:
1: luks2
Key: 512 bits
Expand Down Expand Up @@ -196,6 +257,260 @@
Digest: 5c 30 5b f3 59 db fe 6a 71 c4 9a a0 2d 22 cf 6b
18 e7 cc 8d 6a 44 c9 67 97 f8 34 80 96 69 53 7b

.. note::

As long as the TPM 2.0 emulation storage is not deleted, you will be

Check warning on line 262 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 262, "column": 4}}}, "severity": "INFO"}
able to reboot your QEMU image since the key will persist.


Implementation Details for OP-TEE PKCS#11 Support
-------------------------------------------------

To prevent conflicts with the PKCS#11 token slot utilized by
``aktualizr-lite``, a dedicated slot is necessary.

LmP will set this dedicated slot as **slot 1** with the label ``lmp``.

Before initiating the re-encryption process, the slot is initialized,
and a new **RSA 2048** key is generated. This key never leaves the
PKCS#11 domain.

It is important to emphasize that only the **encrypted master key** is
stored in the LUKS JSON token header area.

Please ensure that you **DO NOT** erase the PKCS#11 token slot or its key
throughout the lifespan of your product. Failure to follow this
precaution will result in the system's inability to boot.

In the event of such a scenario, a recovery key can be created and
provided manually, but it won't support an unattended boot process.

Check warning on line 286 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.Contractions] Avoid Contractions: Use 'will not' not 'won't'.

Raw Output:
{"message": "[Fio-docs.Contractions] Avoid Contractions: Use 'will not' not 'won't'.", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 286, "column": 27}}}, "severity": "WARNING"}


Testing PKCS#11 Support With Qemu (arm64)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Make sure LUKS support is enabled for your ``qemuarm64-secureboot`` target:

.. code-block:: console

$ cat meta-subscriber-overrides/conf/machine/include/lmp-factory-custom.inc
DISTRO_FEATURES:append:qemuarm64-secureboot = " luks"


When running QEMU, please be cautious not to exceed 2GB of memory usage,

Check warning on line 300 in source/reference-manual/linux/linux-disk-encryption.rst

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[Fio-docs.sentence-length] Aim for sentences no longer than 25 words

Raw Output:
{"message": "[Fio-docs.sentence-length] Aim for sentences no longer than 25 words", "location": {"path": "source/reference-manual/linux/linux-disk-encryption.rst", "range": {"start": {"line": 300, "column": 1}}}, "severity": "INFO"}
as attempting to use more than 2GB of memory may prevent the OP-TEE
emulation from successfully booting. So, it's advisable to stay within
this memory limit.

.. code-block:: console

$ qemu-system-aarch64 -m 2048 -cpu cortex-a57 -no-acpi -bios flash.bin \
-device virtio-net-device,netdev=net0,mac=52:54:00:12:35:02 -device virtio-serial-device \
-drive id=disk0,file=lmp-console-image-qemuarm64-secureboot.wic,if=none,format=raw \
-device virtio-blk-device,drive=disk0 -netdev user,id=net0,hostfwd=tcp::2222-:22 \
-object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 \
-chardev null,id=virtcon -machine virt,secure=on -nographic


During the boot sequence, you will observe the following:

.. code-block:: none

[ 1.932467] Freeing unused kernel memory: 4736K
[ 1.933323] Run /init as init process
Starting version 250.5+
[ 53.995060] e2fsck: otaroot: clean, 7841/136880 files, 79834/156064 blocks
Enrolling LUKS2 keyslot based on pkcs11 token
Token successfully initialized
User PIN successfully initialized
Key pair generated:
Private Key Object; RSA
label: luks
ID: 9d
Usage: decrypt, sign
Access: sensitive, always sensitive, never extractable, local
Public Key Object; RSA 2048 bits
label: luks
ID: 9d
Usage: encrypt, verify
Access: local
Engine "pkcs11" set.
Created certificate:
7Certificate Object; type = X.509 cert
label: luks
subject: DN: CN=LmP
ID: 9d
Successfully logged into security token 'lmp' via protected authentication path.
New PKCS#11 token enrolled as key slot 0.
Wiped slot 31.
Successfully logged into security token 'lmp' via protected authentication path.
Successfully decrypted key with security token.
[...]
[ OK ] Reached target Basic System.
Starting D-Bus System Message Bus...
Starting Check and fix an … store of the docker daemon...
Starting IPv6 Packet Filtering Framework...
Starting IPv4 Packet Filtering Framework...
Starting Online LUKS2 disk re-encryption...
Starting User Login Management...
[ OK ] Started TEE Supplicant.
[ OK ] Started Network Name Resolution.
[ OK ] Finished IPv6 Packet Filtering Framework.
[ OK ] Finished IPv4 Packet Filtering Framework.
[ OK ] Starting Network Manager Script Dispatcher Service...
[ OK ] Started Network Manager Script Dispatcher Service.

Linux-microPlatform 4.0.11 qemuarm64-secureboot -

qemuarm64-secureboot login: fio
Password:

fio@qemuarm64-secureboot:~$

[ OK ] Finished Online LUKS2 disk re-encryption.
Starting Resize root filesystem to fit available disk space...
[ 210.434491] EXT4-fs (dm-0): resizing filesystem from 156064 to 160161 blocks
[ 210.448134] EXT4-fs (dm-0): resized filesystem to 160161
[ OK ] Finished Resize root filesystem to fit available disk space.


After the service has finished, you can inspect the volume. First list
the block devices:

.. code-block:: none

fio@qemuarm64-secureboot:~$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
zram0 251:0 0 0B 0 disk
vda 253:0 0 925.6M 0 disk
|-vda1 253:1 0 78M 0 part /var/rootdirs/mnt/boot
|-vda2 253:2 0 200M 0 part /boot
`-vda3 253:3 0 641.6M 0 part
`-vda3_crypt 252:0 0 625.6M 0 crypt /var
/usr
/
/sysroot


Then inspect the encrypted one:

.. code-block:: none

fio@qemuarm64-secureboot:~$ sudo cryptsetup luksDump /dev/vda3
Password:
LUKS header information
Version: 2
Epoch: 99
Metadata area: 16384 [bytes]
Keyslots area: 16744448 [bytes]
UUID: 06be9f40-ac4f-4301-ad33-e566def6023d
Label: otaroot
Subsystem: (no subsystem)
Flags: (no flags)

Data segments:
0: crypt
offset: 16777216 [bytes]
length: (whole device)
cipher: aes-xts-plain64
sector: 512 [bytes]

Keyslots:
1: luks2
Key: 512 bits
Priority: normal
Cipher: aes-xts-plain64
Cipher key: 512 bits
PBKDF: pbkdf2
Hash: sha512
Iterations: 1000
Salt: a2 76 b4 61 3b c6 79 02 1a c1 23 89 02 ca 02 8f
f3 82 ec e6 c4 b0 6a c7 4a 4b 99 5e e6 92 c0 88
AF stripes: 4000
AF hash: sha512
Area offset:32768 [bytes]
Area length:258048 [bytes]
Digest ID: 1
Tokens:
0: systemd-pkcs11
pkcs11-uri: pkcs11:token=lmp;object=luks
pkcs11-key: 38 49 ce f7 3e e9 dc fc 66 3d b8 13 90 ec ec 29
99 73 5d 47 6a cb d0 fc 6c ab 1c a7 26 a8 08 7e
46 b3 5d 15 f5 01 a9 e7 e6 d2 80 72 15 14 0d 0b
61 85 fe ee 1f f8 f0 04 26 c8 46 31 83 52 cc 37
44 d7 2a 83 7d 5a d9 44 a3 90 d0 f5 ff f2 9d e3
6f 09 4b 2c 79 5e df e3 b0 f7 df b4 b2 8c 0b 78
0a 4a 31 c1 d1 63 bb 54 a3 ca c9 a9 a3 88 bc ec
96 68 25 26 75 b3 44 3d 9b ee bc a4 73 a5 e2 b3
f2 5e a3 74 29 32 7a 46 b2 af 55 cf 48 3d b6 ea
4e d0 ca 0c da 06 f1 4e 33 23 73 be bb b0 c0 e1
ab bf 7a 2d f3 d7 7a be 5c 01 e5 d6 ab 43 33 91
48 e7 14 77 61 1c b9 c0 2c 6a 47 36 4c 1f a1 81
39 8c 5b 56 43 fa 86 33 7f 8d ec ee cf 74 1a 3a
43 69 6d bf 3b 70 70 ea 4b f7 02 a0 99 c0 55 02
49 16 14 00 45 da 78 da b9 5e 34 17 65 1b 3b c3
78 26 64 60 bf fe da 11 a0 3b 7a f9 0f 9e 93 8f
Keyslot: 1
Digests:
1: pbkdf2
Hash: sha512
Iterations: 1000
Salt: a6 10 c3 0d 89 22 c4 67 32 c1 c4 49 31 6f 05 10
4a f6 3d bd 7f 26 7a ba 9e 74 54 0b 5f da 54 34
Digest: 58 da 0f b2 ec d5 0d 5d 3d 99 15 85 85 ab e5 40
41 14 9c 57 6a 16 02 08 5d 8f 2a 18 ca 77 2d 7b
e1 be 92 d4 0a 49 f1 f1 77 48 c3 c1 27 35 57 ea
68 47 60 20 15 a1 a2 80 11 c5 dd 8e c7 93 c4 80


You can also examine the PKCS#11 slot created by OP-TEE to verify the
presence of the RSA-2048 key mentioned earlier:

.. code-block:: none

root@qemuarm64-secureboot:/var/rootdirs/home/fio# pkcs11-tool --module /usr/lib/libckteec.so.0 --list-token-slots
Available slots:
Slot 0 (0x0): 94e9ab89-4c43-56ea-8b35-45dc07226830
token state: uninitialized
Slot 1 (0x1): 94e9ab89-4c43-56ea-8b35-45dc07226830
token label : lmp
token manufacturer : Linaro
token model : OP-TEE TA
token flags : login required, PIN pad present, rng, token initialized, PIN initialized
hardware version : 0.0
firmware version : 0.1
serial num : 0000000000000001
pin min/max : 4/128
Slot 2 (0x2): 94e9ab89-4c43-56ea-8b35-45dc07226830
token state: uninitialized

.. note::

The OP-TEE PKCS#11 secure storage emulation will NOT survive across
reboots. As a consequence of this, because the root file system
was encrypted, the system will encounter a failure in mounting the
root file system during the subsequent boot.


If you were to reboot the system under the described circumstances, you
should expect to encounter the following error:

.. code-block :: none

[ 1.776260] registered taskstats version 1
[ 1.776628] Loading compiled-in X.509 certificates
[ 1.879079] Loaded X.509 cert 'Default insecure key from Factory II: 1b2327c0b75d0bc1e4914c8195bbf053629b8abb'
[ 1.902679] uart-pl011 9000000.pl011: no DMA platform data
[ 1.937637] Freeing unused kernel memory: 4736K
[ 1.938472] Run /init as init process
Starting version 250.5+
No slot with token named "lmp" found
PKCS11 certificate not found!


.. _re-encryption:
https://man7.org/linux/man-pages/man8/cryptsetup-reencrypt.8.html

.. _systemd-cryptenroll:
https://www.freedesktop.org/software/systemd/man/systemd-cryptenroll.html

Expand Down
Loading