maint: add a bit more automation to release

[lread]

Currently

I don't see an automated script to trigger a release.

This might be why we are sometimes missing:

• updating the changelog
• updating README with release version

So...

I'll whip up a babashka task that should do releasy stuff like:

1. verify changelog
2. update changelog "unreleased" to release version/date
3. update README
4. push release tag

The release tag will trigger our existing CI release flow.
We've done this on other projects, should not be too hard.

I might also...

See if I can automate creating the GitHub release.
This is something that is easy to forget.

Move to a 1. release? Why not?

I could...

Change the versioning scheme (@borkdude and I seem to prefer 1.0.<release count>).

Any objections?

[borkdude]

I'd love this!

[lread]

changelog

For the changelog, I typically add append a link to the commit compare from last version to the new version into the new version section. Example:

Full commit log

GitHub release

For the GitHub release, I'm thinking a good result would be to have its text point to the specific release anchor in the changelog. Example:

See changelog for details.

As to where to create the GitHub release tag, options:

1. locally - would at least require a GITHUB_TOKEN env var, and depending on how I go, maybe a tool like ghr.
2. on CI - GITHUB_TOKEN would need to be setup (if it is not already) as a secret on CircleCI. CircleCI, I think has the necessary tools.

I think option 2 is more convenient.

I'll play around on a muckabout project I use for this kind of thing.

[lread]

Also: are we married to CircleCI here? GITHUB_TOKEN is already available on GitHub Actions.

[borkdude]

I'm not married to it, but I also don't see why we would change things significantly. cc @slipset

[slipset]

Not married to, but engaged :) There has been a significant gpg and yaml wrangling that's been done to get the release stuff going. I'd not be very motivated for fixing another setup, so if it ain't borked, bork it!

[borkdude]

😇

[lread]

Ah yes, GPG stuff... Ok, if we want to preserve that, let's stick with CircleCI.

And @borkdude, you are an 😇 and an 😈.
But seriously, I think the Clojure Core team's approach makes total sense for them, but I often am OK with risking changes to improve maintainability or to reduce friction/frustration when adding a feature.

On that CI YAML wrangling front, @slipset did you notice #40 and #46? Seems good to me, but maybe you have other opinions?

I'll take full responsibility for fixing anything I bork in this area.

[borkdude]

@lread We could just have both CIs. This is what I do for bb too, in fact, I'm using Circle, Actions, AppVeyor and Cirrus CI there.

[lread]

@borkdude I think that makes good sense for bb, for redundancy right? And maybe features on one CI that another does not have? That's probably not needed here, so I'm good with sticking to one CI for this project.

[borkdude]

OK then, let's keep the one CI.

[lread]

The two CIs...

Oh... maybe you meant use GitHub Actions just to create that GitHub Release?
That might be an option.
But maybe a bit awkward/non-obvious?

GPG Signing Necessary/Valuable/Worth-the-cost-of-understanding-and-maintaining?

If it were not for the need for gpg signing that is all set up on CircleCI, I'd be tempted just to move all the work over to GitHub Actions.
Why are we gpg signing again? Is there a real value to it?

Muckabout progress

Anyway I'm playing on my muckabout project, here's a glimpse of what I have so far:

$ bb publish

[ TASK publish  ]----------------------------------------------------------------

[ Performing release checks ]----------------------------------------------------
✓ on main branch
✓ no uncommitted code
✓ no unpushed commits
✓ changelog has unreleased section
✓ changelog unreleased section attributes valid
✓ changelog unreleased section has content

[ Calculating versions ]---------------------------------------------------------
From [email protected]:lread/muckabout.git
Release version: 1.1.20
Release tag: Release-1.1.20
Last release tag: Release-1.1.19

[ Updating docs ]----------------------------------------------------------------
Applying version 1.1.20 to readme
Applying version 1.1.20 to changelog

[ Committing changes ]-----------------------------------------------------------
[main 9e379b9] Release: updates for version 1.1.20
 3 files changed, 7 insertions(+), 3 deletions(-)

[ Tagging & pushing ]------------------------------------------------------------
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 8 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 1.14 KiB | 1.14 MiB/s, done.
Total 5 (delta 4), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To github.com:lread/muckabout.git
   af7948b..9e379b9  main -> main
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com:lread/muckabout.git
 * [new tag]         Release-1.1.20 -> Release-1.1.20

Local work done.

[ Remote work ]------------------------------------------------------------------
The remainging work will be triggered by the release tag on CI:
- Publish a signed release jar to clojars
- Creating a GitHub release

TASK publish done.

[borkdude]

If you have strong feelings about using Github Actions instead of CircleCI: my take would be: go ahead. As far as I am concerned, GPG can be dropped. So I'll defer to @slipset.

[lread]

Thanks for the reply and opinion @borkdude!

on signing

@slipset, I want to acknowledge the investment and challenge in getting GPG signing to work.
It does not seem trivial. I tried GPG signing in my muckabout project yesterday but was unsuccessful.

If there is real value to signing, by all means, let's sign.
But if there is no clear benefit, I'd suggest dropping this complexity, it would be one less thing for folks to spend their time trying to figure out.

(For other clj-commons projects, I do not sign: rewrite-clj, etaoin, clj-http-lite.)

rationale on moving to github actions

I want to automate creating GitHub releases as part of publishing a release.

I did have experimental success on my muckabout project in creating a GitHub Release from CircleCI.

I did so with a GitHub personal access token. This is not the approach we'd want to take for clj-commons as it grants access at the user, and not the repo level. We'd need to figure out what model makes sense in granting privileges to GitHub from CircleCI.

I spent time looking into this yesterday, one idea is to create GitHub machine users... and there are other approaches too. And we'd need to figure out who would manage this opsy kind of stuff.

But if I just move CI GitHub Actions, this research and concern go away, freeing me up to work on more interesting (to me) things.

[borkdude]

But if I just move CI GitHub Actions, this research and concern go away, freeing me up to work on more interesting (to me) things.

👍

[borkdude]

Converted to a Discussion so we can reserve Issues for things that are ready to be worked on.

[lread]

Now that we have a deps.edn I'm tempted to turf the project.clj. I find projects that have both can sometimes get out of synch and confuse. Any objections?

Also, I've been typically been putting version in a version.edn of late.
I have noticed some neil-ism of tracking version in deps.edn which I'm also cool with. Any preference?

[borkdude]

I think using the neil-ism would be handy since you can them use neil version patch to bump the patch version.

[lread]

Moving this to a discussion was the right move @borkdude, thanks.

topic: changelog

automate update
I like to automate what I can for a release (because I'm human and forget and make typos).
This includes updating "Unreleased" section heading in the change log.

include date
I noticed that clj-yaml includes the date in its changelog headings.
I've typically not done this, but like it. As a user, it allows me to quickly see how long a release has been in the wild. So I'll have automation include a (utc) date.

optionally include attribute to indicate breaking
The other thing I've tried on other projects is using what I call attributes in the release heading.
Currently I've come up with [breaking] and [minor breaking].
I prefer this over separate breaking changes sections because I find it easier read (and automate).
An unreleased header can optionally include these attributes.
Clj-yaml, won't initially use these attributes, but I'm liking the convention.

Example of what I am thinking of:

Changelog

A release with an intentional breaking changes is marked with:

• [breaking] you probably need to change your code
• [minor breaking] you likely don't need to change your code

Unreleased [minor breaking]

• blah

v1.0.38 [breaking] - 2020-01-03

• blah
• blah

commit log

v1.0.37 - 2007-04-13

• blah
• blah

commit log

[lread]

topic: git release tag
I'm not a huge fan of the current Release- prefix.
I prefer v because it is:

• reusable for other things like change logs
• and reads better for release notice emails from GitHub. Ex. Release v1.0.0 instead of Release Release-1.0.0.

I do prefer v but can live with Release- and will not carry out this change if you oppose it @borkdude.

[borkdude]

Not opposing

[lread]

topic: github release

Just in case anybody cares (it is fine if you don't), here's what I'm playing with for automatically generated release:

I think GitHub has lots of options now for populating the release description, but I thought I'd stick to the basics.
The changelog link points to the appropriate version in the changelog.

[borkdude]

Love that!

[borkdude]

Although I'm kind of used to Changelog to be spelled as one word like here: https://keepachangelog.com/en/1.0.0/

[lread]

Yeah me too! Thanks for noticing! Will change that.

[slipset]

So, my 2 cents.

1. I would really appreciate it if the stuff that builds/deploys stays on Circle and that triggering a release is done by pushing a release tag. What leads up to pushing the release tag, I have no strong feelings about, other than that if that work includes pushing to the repo, it should not run on Circle. I do not want Circle to have access to modifying code on Github.
2. I don't care very much wether a release tag is prefixed with Release- or v or something totally different. I do care, however that this is consistent across the cli-commons projects.
3. I also care that when pushing a release tag, we ensure that Github create a release for us, so that source bundles and such are available to downstream projects that need them.

[slipset]

And I seem to be under the impression that there is some naming conventions that inform GitHub that certain tag-names means we want GitHub to create releases for us. But I'm unaware of if this is true or just something I dreamt.