ci: Fix the CI builds after Github Actions moving ubuntu-latest to 24.04

[martin-g]

Drop more pypy3.9 usages

What is the purpose of the change

(For example: This pull request improves file read performance by buffering data, fixing AVRO-XXXX.)

Verifying this change

(Please pick one of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

• Extended interop tests to verify consistent valid schema names between SDKs
• Added test that validates that Java throws an AvroRuntimeException on invalid binary data
• Manually verified the change by building the website and checking the new redirect

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[martin-g]

@KalleOlaviNiemitalo @zcsizmadia Any ideas how to fix this for ubuntu-24.04:

Testhost process for source(s) '/home/runner/work/avro/avro/lang/csharp/src/apache/codec/Avro.File.XZ.Test/bin/Release/netcoreapp3.1/Avro.File.XZ.Test.dll' exited with error: Process terminated. Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.
2024-10-16T07:40:09.5324651Z    at System.Environment.FailFast(System.String)
2024-10-16T07:40:09.5325787Z    at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()
2024-10-16T07:40:09.5327122Z    at System.Globalization.GlobalizationMode..cctor()
2024-10-16T07:40:09.5328235Z    at System.Globalization.CultureData.CreateCultureWithInvariantData()
2024-10-16T07:40:09.5329310Z    at System.Globalization.CultureData.get_Invariant()
2024-10-16T07:40:09.5330141Z    at System.Globalization.CultureInfo..cctor()
2024-10-16T07:40:09.5330853Z    at System.String.ToUpperInvariant()
2024-10-16T07:40:09.5331693Z    at System.Diagnostics.Tracing.EventSource.GetGuid(System.Type)
2024-10-16T07:40:09.5333184Z    at System.Diagnostics.Tracing.EventSource..ctor(System.Diagnostics.Tracing.EventSourceSettings, System.String[])
2024-10-16T07:40:09.5334978Z    at System.Diagnostics.Tracing.EventSource..ctor()
2024-10-16T07:40:09.5336461Z    at Microsoft.VisualStudio.TestPlatform.CoreUtilities.Tracing.TestPlatformEventSource..ctor()
2024-10-16T07:40:09.5338503Z    at Microsoft.VisualStudio.TestPlatform.CoreUtilities.Tracing.TestPlatformEventSource..cctor()
2024-10-16T07:40:09.5340327Z    at Microsoft.VisualStudio.TestPlatform.CoreUtilities.Tracing.TestPlatformEventSource.get_Instance()
2024-10-16T07:40:09.5341999Z    at Microsoft.VisualStudio.TestPlatform.TestHost.Program.Main(System.String[])

?

[KalleOlaviNiemitalo]

Microsoft ended .NET Core 3.1 support on December 13, 2022., Can Apache Avro just stop building for it and testing on it? The Supported Distributions table shows .NET Core 3.1 was supported on Ubuntu 20.04 and 22.04, but not 23.10 nor 24.04.

avro/.github/workflows/test-lang-csharp.yml

Lines 45 to 53 in 290ce8e

	- name: Install .NET SDKs
	uses: actions/setup-dotnet@v4
	with:
	dotnet-version: |
	3.1.x
	5.0.x
	6.0.x
	7.0.x
	8.0.x

avro/lang/csharp/common.props

Lines 38 to 45 in 290ce8e

	<PropertyGroup Label="Target Frameworks">
	<!-- Exe -->
	<DefaultExeTargetFrameworks>netcoreapp3.1;net5.0;net6.0;net7.0;net8.0</DefaultExeTargetFrameworks>
	<!-- Library -->
	<DefaultLibraryTargetFrameworks>netstandard2.0;netstandard2.1</DefaultLibraryTargetFrameworks>
	<!-- Unit Tests -->
	<DefaultUnitTestTargetFrameworks>$(DefaultExeTargetFrameworks)</DefaultUnitTestTargetFrameworks>
	</PropertyGroup>

.NET 5.0 and .NET 7.0 are likewise out of support; in contrast, .NET 6.0 is a Long Term Support release and is still supported. But if there is currently no problem testing on .NET 5.0 and .NET 7.0, then I suppose they can be kept on a best-effort basis.

[KalleOlaviNiemitalo]

The Dependencies list shows "libicu74 (for 24.04 or later)", and libicu-dev on Ubuntu 24.04 also depends on that.

The .NET Core 3.1.32 code that loads ICU libraries seems to be at https://github.com/dotnet/coreclr/blob/v3.1.32/src/corefx/System.Globalization.Native/pal_icushim.c. It checks the CLR_ICU_VERSION_OVERRIDE environment variable; perhaps that could be used for forcing .NET Core 3.1 to load libicu74. But I don't know whether these versions of ICU have compatible ABI for the functions and structures that .NET uses.

[KalleOlaviNiemitalo]

It looks like .NET uses only C APIs of ICU, not C++ APIs. The functions are listed in https://github.com/dotnet/coreclr/blob/v3.1.32/src/corefx/System.Globalization.Native/pal_icushim.h.

Based on the ICU Binary Compatibility documentation, I think the ABI will be compatible if CLR_ICU_VERSION_OVERRIDE=74 is used to force loading libicuuc.so.74 and libicui18n.so.74, which exist in the libicu74 package. And the code in pal_icushim.h attempts to load a range of 20 higher major versions anyway.

[zcsizmadia]

Sorry for being late to the convo. As @KalleOlaviNiemitalo said the isse is the libucu. I wasn't aware that 3.1 cannot run at all on ubuntu 24.04 since it is so old. The only reason all the EOL net targets I still kept in the supported frameworks was to be paranoid when it comes to testing.were. It is totally safe to remove the netcoreapp3.1 support (or any of the others which are not getting supported on newer OSs).

[kojiromike]

For doc/ref:

There are a slew issues referenced at actions/runner-images#10636

GitHub's runner image for ubuntu-24.04 is just plain broken for Python as of now. (As opposed to just backwards-incompatible to how we were doing things).

[zcsizmadia]

Just checked netcoreapp5.0, and even the plain compiler it is not working on a plain ubuntu 24.04. SO 3.1 and 5.0 should be removed (which is ok), to be able to run on ubuntu 24.04 runner.

[martin-g]

As you may noticed I set ubuntu-22.04 for the C# workflows to make it work again.
Feel free to open a new PR for dropping the old .NET versions.

For Python I used python3 -m pip install --break-system-packages ... because the CI uses short-lived VMs anyway and one should not bother with polluting the system namespace. Again feel free to improve if you see a better way!

[zcsizmadia]

I will have a PR soon.

[zcsizmadia]

#3214

[kojiromike]

For Python I used python3 -m pip install --break-system-packages ... because the CI uses short-lived VMs anyway and one should not bother with polluting the system namespace. Again feel free to improve if you see a better way!

This is fine for actions, IMO.