Add Entropy Source and DRNG Manager (ESDM) RNG support

[thillux]

ESDM is a Linux-based user-space PRNG daemon, with configurable entropy sources.

See: https://github.com/smuellerDD/esdm

It currently gets used, when a high level of control over entropy sources is desirable, e.g. for VPN appliance solutions.

In order to use this source, the ESDM server daemon has to be running and botan must be configured --with-esdm.

ESDM currently works only on Linux.

[thillux]

cc @smuellerDD

[thillux]

I tested the FFI-Bindings against https://github.com/thillux/botan-rs/tree/esdm-rng.

e.g. the following worked as expected:

let mut rng = RandomNumberGenerator::new_esdm_pr().unwrap();
for _ in 0..10 {
    let bytes = rng.read(32);
    println!("{bytes:x?}");
}

[thillux]

@reneme I reformatted the code with clang-format, after looking at the CI results.

[coveralls]

coverage: 91.063% (-0.01%) from 91.076%
when pulling 4574d17 on thillux:esdm-rng
into 679ca89 on randombit:master.

[reneme]

I reformatted the code with clang-format, after looking at the CI results.

Thanks! FYI: There are some Emacs, VS Code and .editorconfig config files in src/editors. In case you use any of this, you might want to symlink them into the repo's root dir and benefit from things like "format on save".

[reneme]

This seems interesting and the code looks good already. Thanks for the contribution! I've left a few suggestions down below.

Is there a way to test this in CI? I'd hope that we could just install and configure the daemon in src/scripts/ci/setup_gh_actions.sh and run a few basic smoke tests against it in the unit tests.

[randombit]

So the change already looks pretty good, I think reneme already hit most of the points I would raise on it, but I am going to have to be a downer here and ask

Can you expand more on where and how ESDM is used? It does not seem to be widely distributed, eg per https://repology.org/project/esdm/versions it is only included in AUR and nix (the Debian experimental listing is a name collision with another package). In general our integrations with other systems have been for very widely deployed (eg TPM, PKCS11, OS provided interfaces, or widely deployed libs like zlib or sqlite3) and I'm hesitant to accept something that is so niche.

[mouse07410]

I'm pretty sure it doesn't work on Mac, for example. No idea if it runs on Windows.

[thillux]

So the change already looks pretty good, I think reneme already hit most of the points I would raise on it, but I am going to have to be a downer here and ask

Can you expand more on where and how ESDM is used? It does not seem to be widely distributed, eg per https://repology.org/project/esdm/versions it is only included in AUR and nix (the Debian experimental listing is a name collision with another package). In general our integrations with other systems have been for very widely deployed (eg TPM, PKCS11, OS provided interfaces, or widely deployed libs like zlib or sqlite3) and I'm hesitant to accept something that is so niche.

You're right with ESDM being niche from a major Linux distribution point of view. I cannot weaken/invalidate this argument.

ESDM is currently mostly used under the public radar I would say. My company uses it to provide randomness in a regulated/certified market, on an installed base of >= 100k Linux-based notebooks and plans to use ESDM in more products in the future.

For such a usage in a regulated market or for governmental customers, it offers the following key benefits from my point of view:

• In combination with ESDM Botan can offer a NIST SP800-90B/C compatible RNG
• ESDM offers more control over configured entropy sources, when needed, without the need for a developer to change Botan
• ESDM can also use Botan for its internal cryptographic primitives, so less certification/code review work necessary for the developer
• ESDM is currently the only way to conform to the upcoming NTG.1 definition of the AIS 20/31 2024 from BSI in Germany
• It is easier to comply with changing governmental requirements in user-space in ESDM, than in kernel space (e.g. change /dev/random)

[thillux]

I'm pretty sure it doesn't work on Mac, for example. No idea if it runs on Windows.

That's correct, it runs on Unix/Linux.

[thillux]

I started working on your suggestions and will squash them together again, when finished.

[thillux]

This seems interesting and the code looks good already. Thanks for the contribution! I've left a few suggestions down below.

Is there a way to test this in CI? I'd hope that we could just install and configure the daemon in src/scripts/ci/setup_gh_actions.sh and run a few basic smoke tests against it in the unit tests.

How can I test this by myself? I think I'll need some iterations to get this running.

[reneme]

How can I test this by myself? I think I'll need some iterations to get this running.

The repository policy doesn't let first-time contributors run the CI pipeline. You could iterate in a branch on your private fork, though. For a full end-to-end test, unfortunately the process is commit->push->fail->fix->commit 😢

You can, however, run the underlying CI script locally (that assumes that setup_gh_actions.sh -- or something else -- has already prepared the environment). Simply run: src/scripts/ci_build.py shared to run most things the CI would for the 'shared' (library) target.

[mouse07410]

That's correct, it runs on Unix/Linux.

Does Botan aim to only run on Linux? And at that, on one (possibly major) distribution of Linux?

[thillux]

That's correct, it runs on Unix/Linux.

Does Botan aim to only run on Linux? And at that, on one (possibly major) distribution of Linux?

The ESDM_RNG class is added as optional dependency. On Windows and/or MacOS it just can't be used. This PR does not limit Botan to only Linux.

[thillux]

I'll probably try to add the test this weekend or start of next week.

[reneme]

One more suggestion for our little initialization-concurrency rabbit hole 🕳️, otherwise looks good to me now. Looking forward to a test.

Regarding the utility of this: I tend to agree that this is rather niche. Nevertheless, this has sparked some interest in our company as well, for similar reasons as pointed out by @thillux. FWIW.

[thillux]

@reneme Smoke tests added and PR squashed again.

[thillux]

I'm maybe too stupid to get esdm-server started for the coverage and sanitizer targets. I'll pushed a fix that hopefully works.

[thillux]

All tests green in my repo now.

[thillux]

I started rebasing, will push an updated version for 3.7 soon.

[thillux]

@reneme @randombit CI was green on my testing branch yesterday.