TODO: test reproducibility of packages

[kit-ty-kate]

We could install a package once, save what was installed, reinstall it and then check if there are any differences between the two.

[kit-ty-kate]

Would https://github.com/roburio/orb be easy enough to integrate in a containerized step? @hannesm @reynir
Would i just need to install orb and call orb <pkg>? Are there requirements?

[hannesm]

Hmm, it is a bit more intricate I suspect. There are two questions: which variance of reproducible builds would you like to check (see https://tests.reproducible-builds.org/debian/index_variations.html -- e.g. user id, lang setting, ...), and how much work you'd like to do in the CI.

Orb, as linked to above, does minimize the environment variables that are defined for each build and this way achieves reproducibility (similar to guix/nix) - e.g. even the build path is the same. Earlier versions (rjbou/orb) were slightly different. In addition, orb creates a separate opam switch (thus compilation time will be rather huge).

I guess for the opam repo CI some middle-path would be sensible (or use images that have the same compiler already installed in different switches). An earlier version of orb also supported to use an existing switch (the current version we're using does always create a fresh opam switch).

Options for orb (useful in a CI setting):
--twice (to build it twice)
--diffoscope (to execute diffoscope if there is a difference)

The SOURCE_DATE_EPOCH is set either to the date passed via --date or using Unix.time () by default (the very same SOURCE_DATE_EPOCH is used for both builds if --twice is provided).

[kit-ty-kate]

Somewhat related to #212