Adding CI for building the waterbox and as many emulator cores as possible

[SergioMartin86]

This a proposal to add Github Actions to build all waterboxed (and perhaps also the ported cores) to ensure they still compile correctly.

Check if completed:

• I have run any relevant test suites

• I, the committer, have read the licensing terms for contributors (last updated 2024-03-20) and am compliant

• (perhaps for another PR) add the directly ported cores as well (e.g., quickerNES)

[Morilli]

This looks generally good; I do think we want to have consistent, reproducable builds for waterbox cores especially.

A couple immediate thoughts:

• we probably want a way to not build mame. Not sure if it's worth to have a separate job for each individual core, but at the very least there should be a distinction of mame / not mame
• you're setting the env variable a lot, it should ideally only be required to set CC for the musl build step; everything beyond should still work correctly
• I wasn't sure whether we'd want to setup some kind of caching for the musl/libcxx build before, but looking at the build times from those jobs, it seems to complete in <2 minutes anyways? So I guess that won't be necessary.
• I noticed this in the libcxx build logs:

Run ./do-everything.sh
Submodule 'waterbox/llvm-project' (https://github.com/llvm/llvm-project.git) unregistered for path '../llvm-project'
Submodule 'waterbox/llvm-project' (https://github.com/llvm/llvm-project.git) registered for path '../llvm-project'

• ^ this sounds like it's re-cloning the llvm-project submodule using the setup-llvm.sh, which it shouldn't need to do... might investigate myself later
• Would be nice to a have build artifacts containing the built waterbox cores. I could envision a future where we have core binaries built by CI and be retrievable by some other way than being rebuilt and committed to the main repo every time.

[YoshiRulz]

I'll just leave these here https://gitlab.com/YoshiRulz/yoshis-hawk-thoughts/-/issues/10 https://gitlab.com/TASVideos/BizHawk-ports-CI #2423

[SergioMartin86]

This looks generally good; I do think we want to have consistent, reproducable builds for waterbox cores especially.

A couple immediate thoughts:

• we probably want a way to not build mame. Not sure if it's worth to have a separate job for each individual core, but at the very least there should be a distinction of mame / not mame
• you're setting the env variable a lot, it should ideally only be required to set CC for the musl build step; everything beyond should still work correctly
• I wasn't sure whether we'd want to setup some kind of caching for the musl/libcxx build before, but looking at the build times from those jobs, it seems to complete in <2 minutes anyways? So I guess that won't be necessary.
• I noticed this in the libcxx build logs:

Run ./do-everything.sh
Submodule 'waterbox/llvm-project' (https://github.com/llvm/llvm-project.git) unregistered for path '../llvm-project'
Submodule 'waterbox/llvm-project' (https://github.com/llvm/llvm-project.git) registered for path '../llvm-project'

• ^ this sounds like it's re-cloning the llvm-project submodule using the setup-llvm.sh, which it shouldn't need to do... might investigate myself later
• Would be nice to a have build artifacts containing the built waterbox cores. I could envision a future where we have core binaries built by CI and be retrievable by some other way than being rebuilt and committed to the main repo every time.

Thanks for the suggestions. I think I addressed most, if not all, of these points.

[Morilli]

I do want to get this in master, if just for testing purposes. Couple nitpicks as this doesn't actually run correctly right now, but aside from that this looks good enough for now.

[Morilli]

In the current state, this workflow should now automatically run on every PR or push that modifies any waterbox/* file, and can alternatively be run on demand. Artifacts for the waterbox build itself, all waterbox cores and mame+quicknes will also be uploaded.

I think this is fine and can be merged as is, but I'm open to constructive feedback on whether having mame build that often is fine or not. If nothing comes up I'll probably just merge this later and see how it goes.

[vadosnaprimer]

Are there any caps to potentially hit when building too much? MAME is theoretically too big (thankfully not as giant as non-arcade MAME), but if the wbx workflow changes it makes sense to rebuild them all too.

[SergioMartin86]

Are there any caps to potentially hit when building too much? MAME is theoretically too big (thankfully not as giant as non-arcade MAME), but if the wbx workflow changes it makes sense to rebuild them all too.

I truly don't see a downside to building MAME every single time. GitHub CI is free, fast and parallel.

Ps. As far as I know there are no such caps. If there are, they must be super high to avoid abuse.