Adopt scala library release GHA workflow

[emdash-ie]

Following the example from play-secret-rotation, rather than set up a new developer credential to release the next version of this library, this change sets up guardian/gha-scala-library-release-workflow.

[emdash-ie]

I have removed branch protections, but I don’t think I have the required github access to grant this repo access to the necessary secrets – I will try and find out who does.

[rtyley]

I'm going to review this PR now - may have a few small extra changes to make, the best example PR for adopting gha-scala-library-release-workflow at the moment is guardian/facia-scala-client#299, I'll add the appropriate changes that I see...

I have removed branch protections, but I don’t think I have the required github access to grant this repo access to the necessary secrets – I will try and find out who does.

You'll need a PR like https://github.com/guardian/github-secret-access/pull/21 (but for guardian/marley rather than guardian/facia-scala-client) to get the access automatically added!

[rtyley]

These are a few more tweaks I've added with d26aa63:

• Delete the entire sonatype.sbt file that was providing keys that gha-scala-library-release-workflow now takes care of for us:
  • sonatypeProfileName
  • publishTo - see Adopt GHA Scala Library Release Workflow facia-scala-client#299 (comment)
  • scmInfo & pomExtra - see Adopt GHA Scala Library Release Workflow facia-scala-client#299 (review)
  • homepage
• Specify Java 11-compatible bytecode

As part of some of these changes, I've also removed the vestiges of the build-configuration that adapted the build for Scala 2.11 & 2.12 (eg the versionDependent() helper method) - these are no longer need since #78 removed support for Scala 2.12 in January 2023.

[rtyley]

I think this is good to go now, you just need to raise a PR on https://github.com/guardian/github-secret-access to get the credentials added!

[rtyley]

I can see this made a ✨ successful release with v0.2.19 ✨ - well done!

[emdash-ie]

I can see this made a ✨ successful release with v0.2.19 ✨ - well done!

I had a question about that actually: there’s a github release with the right version number, but I can’t see that version in maven. Does that suggest something’s not working correctly?

I saw that in the sonatype release step there’re a lot of mentions of “marley 0.1.0-SNAPSHOT”, and wondered if there was a problem picking up the version.

Hmm, thinking about it now I looked for it on sonatype and found https://central.sonatype.com/artifact/com.gu/marley_2.13, which looks promising! I don’t really know enough about how maven and sonatype interact, maybe there’s a delay in indexing.

[emdash-ie]

Also, I have been assuming that https://mvnrepository.com/ is an “official” maven repository or something like that, which it doesn’t appear to be. I’ll have to do some reading. Since we can see the dependency on https://central.sonatype.com, I assume everything’s fine.

[rtyley]

I had a question about that actually: there’s a github release with the right version number, but I can’t see that version in maven. Does that suggest something’s not working correctly?

I can see what you mean, and I think something is wrong, probably with Maven Central itself, and perhaps not in a crucial way.

Maven Central, and how things get on to it

When we publish a library, we're ultimately trying to get the artifacts onto Maven Central, which is the biggest, oldest, and most well-known repository for Java artifacts. Maven is a build tool for Java that was created around 2001-2004. If your artifacts are on Maven Central, nobody needs any additional configuration to grab those artifacts - Maven Central is the default place to get artifacts.

We can't upload to Maven Central directly (Maven Central is like a big read-only immutable-but-growing repository) and instead we upload to oss.sonatype.org, and once checks are successfully performed there, oss.sonatype.org will 'promote' the new files to Maven Central (the sync process for that generally takes ~10 minutes, but occasionally longer).

All of these systems are run by Sonatype, a company started in 2008 which has stewardship of Maven Central:

• Maven Central
• oss.sonatype.org - our route for getting onto Maven Central
• search.maven.org - searches what's on Maven Central

mvnrepository.com is not associated with Sonatype, as far as I know, but is a very well-known alternative to search.maven.org put together by independent developer Fernando Rodriguez Olivera. I personally don't use it, I think it got popular when search.maven.org had a period of bad stability or missing features.

As an aside, as a Scala developer it's worth being aware of the Scaladex at index.scala-lang.org, because it's got a much better understanding of Scala's language-version-specific artifacts:

What's going wrong with the release of Marley 0.2.19?

It looks like the release succeeded, at least as far as the artifacts being published on oss.sonatype.org:

The release is even recognised in the Scaladex and Maven Central search:

The weird part is only when we come to browse at the raw files available on Maven Central:

• https://repo1.maven.org/maven2/com/gu/marley_2.13/0.2.19/ - works, we can see the files there
• https://repo1.maven.org/maven2/com/gu/marley_2.13/ - denies that the above folder ('0.2.19') exists!

Looks like something just a bit flakey in the CDN or webserver responsible for https://repo1.maven.org/maven2/com/gu/marley_2.13/ ?!

What's the fix?

I think it's probably worth just kicking off another release to see what happens?!

What about the weird '0.1.0-SNAPSHOT' in the logs?

I saw that in the sonatype release step there’re a lot of mentions of “marley 0.1.0-SNAPSHOT”, and wondered if there was a problem picking up the version.

As it happens, in gha-scala-library-release-workflow the logs for 🔒 Sonatype Release job currently always mention 0.1.0-SNAPSHOT, no matter what version you're publishing (eg see this here for the etag-caching library, which was publishing v2.0.5), but this is only because the version number printed here has no technical effect.

The version number is baked into the artifacts in the earlier 🎊 Create artifacts job, which basically produces a big folder full of jars, and the 🔒 Sonatype Release job then takes that folder and uploads it a single big 'bundle' HTTP upload - it doesn't actually need to know the version number at that step, so I hadn't bothered to configure it, but I agree it's confusing, and for that alone it should probably be fixed.

[rtyley]

Looks like something just a bit flakey in the CDN or webserver responsible for https://repo1.maven.org/maven2/com/gu/marley_2.13/ ?!

Note also that I can break-the-cache - and therefore fix-the-page - by adding a random query parameter to the url:

https://repo1.maven.org/maven2/com/gu/marley_2.13/?foo=bar - WORKS

https://repo1.maven.org/maven2/com/gu/marley_2.13/ - still fails for everyone

Probably the de-caching of this url failed?

[emdash-ie]

Ah, thank you for the very thorough explanation!

I think it's probably worth just kicking off another release to see what happens?!

As in: run the release workflow again, which I suppose will generate v0.2.20 which is identical to v0.2.19? I’m not sure I like the sound of that – I guess it’s not particularly important, but I’d like each release to differ. (Unless running it again with no changes will just redeploy the same release? But then I’m not sure about that either.)

but this is only because the version number printed here has no technical effect.

Ah yes, I wondered if that was the case.

I agree it's confusing, and for that alone it should probably be fixed.

Would it be worth setting the version string to “this-version-string-is-irrelevant”? (If that’s possible.) Having the correct number there could also be confusing if it suggests the number matters.

[rtyley]

As in: run the release workflow again, which I suppose will generate v0.2.20 which is identical to v0.2.19? I’m not sure I like the sound of that – I guess it’s not particularly important, but I’d like each release to differ.

Oh, you would not like how I've been testing gha-scala-library-release-workflow 😀

Right now, thanks to Scala Steward's batching settings, I personally don't feel too bad about releasing lots of 'version-bump-patch' releases that are completely empty - I don't want to create too much noise for my fellow devs, but that's about my only concern, and I do need to do lots of releases to check the release workflow. Considering things didn't go perfectly with v0.2.19 I'd be tempted to do another one, just to get a completely successful run.

Still, so long as you can successfully use v0.2.19, I don't think that clean slate of a release run is really that important!

Would it be worth setting the version string to “this-version-string-is-irrelevant”? (If that’s possible.) Having the correct number there could also be confusing if it suggests the number matters.

That's a good point - it looks like the description string can be set with the sonatypeSessionName sbt key, so it could be set to something reasonable. I'm just not entirely sure if it's in some way desirable to have a consistent key, or if the key really should have the correct version in, because there might be some relevant recovery/clean-up logic associated with it...

Finally fixed in guardian/gha-scala-library-release-workflow#29

[emdash-ie]

As in: run the release workflow again, which I suppose will generate v0.2.20 which is identical to v0.2.19? I’m not sure I like the sound of that – I guess it’s not particularly important, but I’d like each release to differ.

Oh, you would not like how I've been testing gha-scala-library-release-workflow 😀

Oh no 😅

Right now, thanks to Scala Steward's batching settings, I personally don't feel too bad about releasing lots of 'version-bump-patch' releases that are completely empty - I don't want to create too much noise for my fellow devs, but that's about my only concern, and I do need to do lots of releases to check the release workflow. Considering things didn't go perfectly with v0.2.19 I'd be tempted to do another one, just to get a completely successful run.

Ok, that’s a good point – I’ll do another release.