GHA workflows - are we doing too much CI testing?

[colinleach]

The Julia track currently tests 3 versions of Julia against 3 OS's, which seems unusual compared with similar tracks. This is the relevant part of exercise-tests.yml:

  test:
    name: Julia ${{ matrix.julia-version }} - ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        julia-version: ["1.6", "1", nightly]
        os: [ubuntu-22.04, windows-2022, macos-14]
        exclude:
          - julia-version: 1.6
            os: macos-14

Python and R both seem fine with just using ubuntu-22.04. Even C# uses only ubuntu-20.04, despite its Windows heritage.

I'm particularly keen to get rid of the Windows/Julia 1.6 combination, which is glacially slow (15-20 minutes, compared with <5 min for all the others). That should go away in any case later this year, when 1.6 is due to be dropped as the LTS version, but maybe Windows and Mac are now inappropriate?

I don't know the history of this choice, but perhaps it made more sense when Julia was less mature and stable, so there could be important differences between platforms.

Thoughts?

[depial]

I'm unaware of the reasons for testing all the OSs, but did notice the issue with Windows a while ago (if I remember correctly it hangs longest when loading dependencies).

We did have an issue with MacOS and Julia 1.6 a few months ago which @cmcaine believed was due to 1.6 not supporting ARM versions of MacOS. There's a bit more detail in PR #733 if you want to have a look.

[colinleach]

We did have an issue with MacOS and Julia 1.6 a few months ago

Presumably, that explains the exclude: in the testing. It still leaves me wondering why we test Mac and Windows, when other tracks don't.

I asked about this on Discord yesterday but with no replies so far.

[BNAndras]

The likely answer to me is the boring one. Most tracks don’t bother testing multiple platforms since they don’t have to. Exercism exercises generally don’t go in directions where platform differences may become significant like I/O or networking. Linux is the default platform to test since most if not all test runners run Linux.