Unexpected results from several recent releases

[willmurphyscode]

What happened:

We recently made several release notes via chronicle whose release notes later needed to be handed edited.

• https://github.com/anchore/vunnel/releases/tag/v0.15.1 is missing fix: respect "not applicable" tag in Mariner Linux provider vunnel#244
• https://github.com/anchore/grype/releases/tag/v0.64.2 erroneously included grype v0.64.0 template zeroes the template content grype#1388, and I had to add changelog-ignore, run the release, and then remove it
• https://github.com/anchore/grype/releases/tag/v0.64.1 erroneously omitted grype v0.64.0 template zeroes the template content grype#1388, and it had to be added by hand to the release notes

What you expected to happen:

Two expectations were not met here:

1. The changelog was initially generated with some error (see above)
2. In 2 of these cases, the changelog looked correct locally, but then re-introduced the error when run in CI.

How to reproduce it (as minimally and precisely as possible):

I'm not sure; chronicle doesn't support regenerating release notes for an existing release

Anything else we need to know?:

The vunnel release closed an issue in a different repo, which seems to have confused chronicle, likely because the GH token I was using had minimal permissions, and probably couldn't read issues on the other repo.

The two grype releases were in pretty rapid succession.

Environment:

• Output of chronicle version:

[willmurphyscode]

https://github.com/anchore/yardstick/releases/tag/v0.8.0 is another release that needed release notes hand-edited.

[wagoodman]

#107 has addressed one of the issues causing problems here, but there is another failure mode too.

Take the v0.15.1 vunnel release:

[0000] DEBUG found existing tag="v0.15.1" at HEAD which does not have an associated release
[0000]  INFO until tag=v0.15.1
[0000]  INFO since release-timestamp=2023-07-11 16:44:56 UTC tag=v0.15.0
...
[0002] TRACE PR #244 filtered out: merged at or after 2023-07-18 20:25:25 UTC (merged 2023-07-18 20:25:26 UTC)

The notable line is the last one, which is a hit on the prsAtOrBefore function

chronicle/chronicle/release/releasers/github/summarizer.go

Line 571 in cc4be36

filters = append([]prFilter{prsAtOrBefore(untilTag.Timestamp.UTC())}, filters...)

... which is passing the timestamp from the git tag (from the "until" tag).

Take note that the PR merged is pretty close to the cutoff. This is what is causing the problem -- when using lightweight git tags then the time you tag is not saved, it will use the timestamp from the commit when it merges to main. However, it's possible (probable) that the github backend will only record the PR as merged after the commit has been created.

There are a few answers here (not mutually exclusive):

1. Use annotated tags (Support annotated tags #35 ) in which case the timestamp from the underlying commit object that represents the tag info would be used instead of the commit object that the tag points to (like with a lightweight commit). We can warn the user of potentially inaccurate results when non-annotated tags are in use.
2. Use purely graph analysis for determining PR inclusion, remove all timestamp-based approaches
3. Pad the "until" timestamp with an allowable buffer (maybe even configurable).

The same failure mechanism exists for linked PRs and issues: you can have a fixes #issuenum within a PR description, merge the PR, and the issue is closed after the PR merges. Occurring approximately here:

chronicle/chronicle/release/releasers/github/summarizer.go

Line 287 in cc4be36

prFilters = append(prFilters, prsAtOrBefore(untilTag.Timestamp.UTC()))

[wagoodman]

Given that #109 is ready, I think it would be easiest to swap the consumer workflows to use annotated tagging.

[wagoodman]

I think #109 solved all of these issues

[wagoodman]

manually added to the last release notes -- adding label so it wont show up in the next changelog since it was closed after the fact