Avoid build-failures due to missing service-version segment increment

[HannesWell]

A common problem contributors of Eclipse projects face are build failures due to missing service-version segment increments.
The build then fails with messages like

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.eclipse.tycho.extras:tycho-p2-extras-plugin:3.0.0-SNAPSHOT:compare-version-with-baselines
(compare-attached-artifacts-with-release) on project org.eclipse.tips.feature: Only qualifier changed for
(org.eclipse.tips.feature.feature.jar/0.2.1700.v20220619-1602). Expected to have bigger x.y.z than what is available in baseline
(0.2.1700.v20220220-0850) -> [Help 1]

This is annoying and time-consuming for contributors and build servers because it leads to (at least) one more build to fix the version.
And even if one tries to anticipate such failures and checks if the version has already been bumped within a release cycle, this is an tedious, time consuming task.

There were already other attempts to reduce the response time (like eclipse-tycho/tycho#1018), but I would like to reduce the risk for build failure due to missing service-version segment increments to a minimum, maybe even zero.

---

My suggestion to tackle this problem is to enhance PDE's API-Tools to also detect the need for service version-increments.
Then one would get already get an error in the IDE indicating that the service version has to be incremented, just like one gets when a minor or major version bump is necessary. To enable this, API-Tools needs to know the build-qualifier of a Plug-in project and can then check if the build-qualifier changed in comparison to the baseline.

With recent changes in M2E starts to support polyglot projects (in the context of eclipse-m2e/m2e-core#119 or with eclipse-m2e/m2e-core#791) so M2E could compute the (expected) build-qualifier of a project while considering all configuration of the Tycho jgit-timestampt provider (like file exclusions, etc.). One drawback would be that it only works after a commit (unless M2E assume the build-qualifier is 'now' for files detected as changed by git). Another drawback would be time-zones (not sure if this a real issue since the qualifier of projects in the baseline is usually a bit older than just a few hours).

To let API-Tools know the build-qualifier it could provide an extension-point/OSGi-service for a BuildQualifierProvider and M2E could register such provider.
This imitates one important task of Tycho's compare-version-with-baselines mojo is doing.

---

As an alternative approach @laeubi suggested to compute the service-version during the build:

• Css background handler eclipse-platform/eclipse.platform.ui#110 (comment)
• Create an automatic version bump plugin eclipse-tycho/tycho#1034
  Would we then just use a placeholder just like for .qualifier in the source?

cc @vogella

[laeubi]

I don't think m2e has to do anything here and it would be quite complicated to setup here, because the actual value of the build qualifier is irrelevant.

You actually need two things what could be done in API tools itself:

1. You need the baseline repository configured somewhere (maybe this is already done with the API Baseline set?)
2. You need to compare if anything changed compared to the baseline (as APi tools can warn about API problems itself it should already contain such comparators or one can copy the code from tycho-extras)

Whenever anything in the binary build of a bundle changed (even if not an API problem) and the version is the same as in the baseline a micro-bump is required.

[HannesWell]

Yes the API-tools should be able to to detect any changes to the baseline. At least I already saw messages like 'service-version' increased unnecessarily. So there seems already some general comparision.

But comparing against all files can lead to false positive results because usually some project files excluded from the build-qualifier computation. For example the pom.xml is excluded for (almost) all Eclipse Platform Projects and although more and more poms are removed the parent-version increment in the beginning of each dev-stream would trigger errors stating that the service version has to be incremented.

I don't think m2e has to do anything here and it would be quite complicated to setup here, because the actual value of the build qualifier is irrelevant.

That's right the actual value is irrelevant. But what do you think will be so complicated?
Maybe I'm too naive, but I would have added an Interface like IBuildQualifierProvider that only has a method getBuildQualifier(IProject) or maybe isBuildQualifierDifferent(IProject project, String baselineQualifier) that is get as OSGi service and would provide an implementation of that service in M2E that acts as connector to the tycho-packaging plugin and or jgit timestamp provider that either processes its configuration and computes the qualifier/if it changed or just executes it and reads the resulting qualifier (I suspect that the former is more work but faster).
This of course has to be worked out and it spans two projects but in general I see no blockers or really hard things for this. But I'm happy to get them told, before I waste my time on them. :)

[HannesWell]

To make this really useful the required service-version bumps would also have to be computed for Features.
So API-Tools would have to be enhanced in that direction as well.

[laeubi]

It could be as easy as comparing the xml content for added/removed/changed items to cover the most common cases and comparing for any version change ignoring the qualifier.

[merks]

I think the common case is that one of the included bundles changes its version for whatever normal reason and then the feature is indirectly changed because the included bundle's 0.0.0 version specified in the feature.xml source expands to the included bundle's actual version at build time.

[laeubi]

Yep that's what I mean, PDE can already compute these "workspace versions" and can then compare these (just need to ignore the qualifier) to detect a version bump required.

[mickaelistria]

I don't think PDE can help here as it is not responsible of producing final binaries, and service release requires to compare binaries to decide whether it is necessary to change it or not. For example, just reparenting a pom file so tycho.scmUrl changes, like what happened in equinox, would require to bump service release but PDE cannot be aware of that.

[HannesWell]

That's the reason why I suggest above to let M2E provide something like a BuildQualifierProvider that can consider the exclusions specified in the poms: #165 (reply in thread)

[laeubi]

That's right the actual value is irrelevant. But what do you think will be so complicated?

Just two trivial examples:

1. You are using the timestamped provider --> Qualifier will change every (milli)second/minute/day (depending on the pattern)
2. You are using JGit --> every dirty file will make JGit either fail, ignore or use the value from (1)

Beside that, just assume you fixed that in some smart way ... Qualifier changes are allowed! They re just not allowed (by some eclipse projects!) if, compared to the baseline the artifact has changed, so how do you know it has changed?

So in the end it all boils down having run the (full) build in the background to get a normative answer, you also need to clone the git repositories and guess what the user will check in and then compare the artifacts.

I don't think PDE can help here as it is not responsible of producing final binaries, and service release requires to compare binaries to decide whether it is necessary to change it or not. For example, just reparenting a pom file so tycho.scmUrl changes, like what happened in equinox, would require to bump service release but PDE cannot be aware of that.

Actually PDE can produce binaries, it might not do it exactly as the build but as described above, if we aim at that we need to actually build the artifact what is a way to complicated and I suspect people won't deserve the additional CPU and runtime cost of this.

So it is better to aim at the 80% case here (how often do the scmURL changes?), so PDE already has the class output folder and it knows from the bin.includes what resources will go into the final jar and can compare if any of those has changed, and give a hint that a version bump might be required and let the final decisions to the build, because this will already cover most of the annoying cases we see.