Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: allow building with recent C compilers #72

Merged
merged 3 commits into from
Aug 2, 2024

Conversation

ee7
Copy link

@ee7 ee7 commented Jul 24, 2024

TL;DR: We should be able to build Chalk outside a container without specifying extra compiler options. But this is currently not possible when recent C compilers (Clang: within the last couple of years; GCC: within the last few months) are installed, due to C compilation errors.

Fixing this would make building Chalk:

  • Faster for a Chalk developer who prefers building natively locally
  • More reliable in less reproducible environments

Right now, there's a significant subset of developers who would see errors if they cloned the Chalk repo and ran nimble build.

I suggest that we add three passC lines in nimutils to disable these errors downgrade these errors to warnings, and move on.


Before this PR, nimutils (and therefore Chalk, see crashappsec/chalk#290) could not be built with recent stable releases of Clang and GCC, as some warnings now default to an error. This affected Clang 15.0.0 (2022-09-06) or later, and GCC 14.1 (2024-05-07) or later, and:

  • Made it harder to build Chalk outside of a container during development. Simply writing the options from this PR on the command line after nimble build was not sufficient, because that wouldn't forward the options to the command used to compile e.g. this repo's strcontainer.c or switchboard.c.

  • Would prevent certain OS or compiler version bumps for the builder image.

For now, simply disable these errors downgrade these errors to warnings. Some are in md4c.h, and others will be resolved when we move to libcon4m.

As background, see e.g. the breaking changes section of the release notes for Clang 16:

  • The -Wimplicit-function-declaration and -Wimplicit-int warnings now default to an error in C99, C11, and C17. As of C2x, support for implicit function declarations and implicit int has been removed, and the warning options will have no effect. Specifying -Wimplicit-int in C89 mode will now issue warnings instead of being a noop.
  • -Wincompatible-function-pointer-types now defaults to an error in all C language modes.

and release notes for Clang 15:

  • The -Wint-conversion warning diagnostic for implicit int <-> pointer conversions now defaults to an error in all C language modes.

and the porting guide for GCC 14:

The initial ISO C standard and its 1999 revision removed support for many C language features that were widely known as sources of application bugs due to accidental misuse. For backwards compatibility, GCC 13 and earlier diagnosed use of these features as warnings only. Although these warnings have been enabled by default for many releases, experience shows that these warnings are easily ignored, resulting in difficult to diagnose bugs. In GCC 14, these issues are now reported as errors, and no output file is created, providing clearer feedback to programmers that something is wrong.

Implicit function declarations (-Werror=implicit-function-declaration)

It is no longer possible to call a function that has not been declared. In general, the solution is to include a header file with an appropriate function prototype. Note that GCC will perform further type checks based on the function prototype, which can reveal further type errors that require additional changes.

Using pointers as integers and vice versa (-Werror=int-conversion)

GCC no longer treats integer types and pointer types as equivalent in assignments (including implied assignments of function arguments and return values), and instead fails the compilation with a type error.

Type checking on pointer types (-Werror=incompatible-pointer-types)

GCC no longer allows implicitly casting all pointer types to all other pointer types. This behavior is now restricted to the void * type and its qualified variations.

and the Gentoo wiki page on Modern C porting.

Before this commit, nimutils (and therefore Chalk [1]) could not be
built with recent stable releases of Clang and GCC, as some warnings now
default to an error. This affected Clang 15.0.0 (2022-09-06) or later,
and GCC 14.1 (2024-05-07) or later, and:

- Made it harder to build Chalk outside of a container during
  development. Simply writing the options from this commit on the
  command line after `nimble build` was not sufficient, because that
  wouldn't forward the options to the command used to compile e.g.
  this repo's `strcontainer.c` or `switchboard.c`.

- Would prevent certain OS or compiler version bumps for the builder
  image.

For now, simply disable these errors. Some are in md4c.h, and others
will be resolved when we move to libcon4m.

As background, see e.g.:

- The breaking changes section of the release notes for Clang 16, which
  contains [2]:

      The `-Wimplicit-function-declaration` and `-Wimplicit-int`
      warnings now default to an error in C99, C11, and C17. As of C2x,
      support for implicit function declarations and implicit int has
      been removed, and the warning options will have no effect.
      Specifying `-Wimplicit-int` in C89 mode will now issue warnings
      instead of being a noop.

      -Wincompatible-function-pointer-types` now defaults to an error in
      all C language modes.

- The release notes for Clang 15 [3]:

      The `-Wint-conversion` warning diagnostic for implicit int <->
      pointer conversions now defaults to an error in all C language
      modes.

- The porting guide for GCC 14 [4]:

      The initial ISO C standard and its 1999 revision removed support
      for many C language features that were widely known as sources of
      application bugs due to accidental misuse. For backwards
      compatibility, GCC 13 and earlier diagnosed use of these features
      as warnings only. Although these warnings have been enabled by
      default for many releases, experience shows that these warnings
      are easily ignored, resulting in difficult to diagnose bugs. In
      GCC 14, these issues are now reported as errors, and no output
      file is created, providing clearer feedback to programmers that
      something is wrong.

      #### Implicit function declarations (`-Werror=implicit-function-declaration`)

      It is no longer possible to call a function that has not been
      declared. In general, the solution is to include a header file
      with an appropriate function prototype. Note that GCC will perform
      further type checks based on the function prototype, which can
      reveal further type errors that require additional changes.

      #### Using pointers as integers and vice versa (`-Werror=int-conversion`)

      GCC no longer treats integer types and pointer types as equivalent
      in assignments (including implied assignments of function
      arguments and return values), and instead fails the compilation
      with a type error.

      #### Type checking on pointer types (`-Werror=incompatible-pointer-types`)

      GCC no longer allows implicitly casting all pointer types to all
      other pointer types. This behavior is now restricted to the void *
      type and its qualified variations.

- The Gentoo wiki page on Modern C porting [5].

[1] crashappsec/chalk#290
[2] https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNotes.html#potentially-breaking-changes
[3] https://releases.llvm.org/15.0.0/tools/clang/docs/ReleaseNotes.html
[4] https://gcc.gnu.org/gcc-14/porting_to.html#warnings-as-errors
[5] https://wiki.gentoo.org/wiki/Modern_C_porting
@ee7 ee7 requested a review from miki725 July 24, 2024 10:16
@ee7 ee7 marked this pull request as ready for review July 24, 2024 10:16
@ee7 ee7 requested a review from viega as a code owner July 24, 2024 10:16
Copy link
Contributor

@miki725 miki725 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont understand the full implications of these so please get a 👍 from @viega

didnt nim 2.0.8 already bump the compiler?

@ee7
Copy link
Author

ee7 commented Jul 24, 2024

didnt nim 2.0.8 already bump the compiler?

A bump from Nim 2.0.0 to Nim 2.0.8 only bumps Nim itself, and consequently Nim itself should no longer generate C code for which newer C compilers produce errors by default.

However, before this PR, nimble build would error when such newer C compilers are installed, because they produce errors by default for the C code in this repo that isn't generated by Nim at compile time.

The bump from our ubuntu:2.0.0 to ubuntu:2.0.8 did additionally bump GCC, but only from 11 to 13. That's why bumping to ubuntu:2.0.8 in Chalk didn't produce a CI failure: GCC only errors by default for Chalk from GCC 14 onwards. Clang has errored by default for these C constructs for nearly two years, but we don't currently build Chalk in CI with a sufficiently recent version of Clang to encounter that.

The context there is:

@ee7 ee7 changed the title nimscript: allow building with recent C compilers build: allow building with recent C compilers Jul 24, 2024
@nettrino
Copy link
Contributor

No objections to actually merge this since its happening anyhow, but what is the level of effort required to actually address these in their root cause?

ee7 added 2 commits August 1, 2024 12:56
Turn these back into warnings, even when the C compiler produces errors
by default, rather than suppressing the message completely.
@ee7
Copy link
Author

ee7 commented Aug 1, 2024

For visibility: these C compilation errors have now blocked aarch64 macOS from inclusion in the latest Chalk binaries (0.4.9, 2024-07-30). My understanding is that this should only be due to GitHub updating the runner's default version of Clang, although the release notes make no mention of this. However, I have noticed over the years that sometimes-surprising major version bumps are made in the GitHub runner images, and in particular I do recall Clang major version bumps when pinned to macos-xy.

what is the level of effort required to actually address these in their root cause?

Likely not enormous. However, at least some are from the external markdown parser md4c, and others will disappear once libcon4m is ready and chalk has integrated it.

For libcon4m, I added the most recent stable releases of clang and gcc to CI, and the only -Wno options that we currently pass there are:

  • -Wno-unused-parameter
  • -Wno-cast-function-type
  • -Wno-atomic-alignment (only for the combination of x86_64 + clang)

That is, none of the errors handled by this PR occur in libcon4m, and libcon4m CI enforces that.


In case it was preferable, I've added commit c8e6ef2 so that this PR merely downgrades these errors to warnings, rather than disabling the diagnostic entirely.

@ee7 ee7 requested a review from nettrino August 1, 2024 11:12
@miki725
Copy link
Contributor

miki725 commented Aug 2, 2024

merging to include as part of the next patch release to fix mac os arm builds

@miki725 miki725 merged commit 1f7f51e into dev Aug 2, 2024
2 checks passed
@miki725 miki725 deleted the ee7/allow-building-with-recent-c-compilers branch August 2, 2024 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants