Wolfi is now compliant with OpenSSF recommended C/C++ compiler options (v2024-06-27)

[xnox]

New changes

Prior to November 2024, melange builds in wolfi relied on environment variables to set many C/C++ compiler options.
However, these environment variables are not honored by many build systems. CMake, Bazel, CGO, OpenJDK all consistently able to override or ignore environment variables.

Instead Wolfi is adopting clang configuration files and GCC spec files. Using on disk configuration files ensures that compiler really does use the recommended OpenSSF compiler options.

From November 2024:

• build-*.env is removed
• compilers depend on openssf-compiler-options package
• the package provides GCC spec files, GCC wrappers, Clang config files

Prior to November 2024:

• build-*.env files were providing suggested (but often ignored) compiler environment variables

How to deal with build failures when warnings are treated as errors

1. Attempt build
2. A warning is uncovered and is getting enforced, the code is wrong, but we are not here to police upstream source code - it is likely the source code bug has always existed and yet everyone keeps building and running that code, use appropriate CFLAGS or CXXFLAGS to downgrade from error to warning, for example:

criu/net.c:230:32: error: 'net/unix/' directive output truncated writing 9 bytes into a region of size 0 [-Werror=format-truncation=]

3. Update override (CFLAGS for c, CXXFLAGS for c++ etc)

environment:
  environment:
    CFLAGS: "Wno-error=format-truncation"

4. Go back to step 1, because the build can go further and uncover a different error, e.g.

 criu/tls.c:195:9: error: 'packet' may be used uninitialized [-Werror=maybe-uninitialized] uses=autoconf/make-install

Meaning one may have to update CFLAGS again

environment:
  environment:
    CFLAGS: "Wno-error=format-truncation -Wno-error=maybe-uninitialized"

5. build passed! Open upstream bug report documenting that GCC with -fhardened fails project build, triggering "Werror=format-truncation Werror=maybe-uninialized" (note whichever overrides you had to add) and provide the final build log as attachment, as it will have all net-new warnings printed for upstream to investigate.

Some known examples / anecdotes are listed below. It has been discovered that upstreams that use Werror and have not been tested with GCC-14 with -fhardened flag can trigger the below warnings as errors, thus needing:

• -Wno-error=maybe-uninitialized
• -Wno-error=format-truncation
• -Wno-error=implicit-function-declaration
• -Wno-error=implicit-int

And others.

How to deal with customizing build behaviour

Any and all flags passed via specs and configuration files, can be overridden via CFLAGS as usual, in well behaved build systems, i.e. one can opt-out of pie with -fno-pie, opt out of FORTIFY_SOURCE with undefining it and redefining it to any desired value and so on.

How to really really revert the build to behave as before

This is is a brake glass to escape option.

1. Create a new bug report at https://github.com/wolfi-dev/os/issues/new/choose with the full failing build log, together with the full melange file.

2. Use the following environment block in the melange file

environment:
  environment:
    CFLAGS: "-O2 -Wall -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" 
    CXXFLAGS: "-O2 -Wall -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer"
    CPPFLAGS: "-O2 -Wp,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS" 
    LDFLAGS: "-Wl,--as-needed,-O1,--sort-common -Wl,-z,relro,-z,now,-z,noexecstack"
    GCC_SPEC_FILE: "/dev/null"

If using clang, pass in --config-system-dir=/var/empty to clang, for example via CFLAGS, there is no environment variable equivalent.

[xnox]

Outstanding steps:

• Analyse remaining known build failures and apply appropriate Wno-error flags to them; or complete openssf compiler options bypass
• Make gcc & clang depend on openssf-compiler-options
  • gcc 14 & clang 19 PR Make default gcc & clang compilers use openssf-compiler-options #33213
  • gcc 13 & 12 PR gcc-12 & gcc-13: enable openssf-compiler-options by default #33215
  • clang 15..18 PR clang-15..18: enable openssf-compiler-options by default #33216
• Drop C/C++ flags from build-*.env files
  • PR build-*.env: drop environment variables in favor of openssf-compiler-options #33217
• Declare compliance with OpenSSF compiler hardening guide up to 2024-06-27 edition
• Wait for things to settle down
• Mass commit to drop explicit depends on openssf-compiler-options in all the individual packages
• Evaluate future optimisations and upgrades to 2024-10-31 edition OpenSSF compiler options upgrade 31st October #33058

Force-majour:

• Re-add back build-*.env flags
• Drop gcc & clang depends on openssf-compiler-options
• Add steps here on what got uncovered as not working