Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MBUILDCACHE-103] Allow incremental restore in case of plugin parameter mismatch #177

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

reda-alaoui
Copy link
Member

@reda-alaoui reda-alaoui commented Aug 21, 2024

This PR adds configuration flag maven.build.cache.incrementalReconciliationOnParameterMismatch to allow the following behaviour. If a parameter mismatch is detected, the build will try to complete starting with the mismatched parameter instead of throwing the whole cache entry. At the end, it will save the result.

The correct behaviour of the operation is subordinated to the correct configuration of attachedOutputs parameter.

Orthogonal subject, but I think attachedOutputs should be set to the following values by default:

<attachedOutputs>
      <dirNames>
        <dirName>classes</dirName>
        <dirName>test-classes</dirName>
      </dirNames>
    </attachedOutputs>

Following this checklist to help us incorporate your
contribution quickly and easily:

  • Make sure there is a MBUILDCACHE JIRA issue filed
    for the change (usually before you start working on it). Trivial changes like typos do not
    require a JIRA issue. Your pull request should address just this issue, without
    pulling in other changes.
  • Each commit in the pull request should have a meaningful subject line and body.
  • Format the pull request title like [MBUILDCACHE-XXX] - Fixes bug in ApproximateQuantiles,
    where you replace MBUILDCACHE-XXX with the appropriate JIRA issue. Best practice
    is to use the JIRA issue title in the pull request title and in the first line of the
    commit message.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Run mvn clean verify to make sure basic checks pass. A more thorough check will
    be performed on your pull request automatically.
  • You have run the Core IT successfully.

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

@reda-alaoui reda-alaoui force-pushed the MBUILDCACHE-103 branch 2 times, most recently from 846045a to ee35226 Compare August 22, 2024 12:28
for (MojoExecution cacheCandidate : cachedSegment) {
if (cacheController.isForcedExecution(project, cacheCandidate)) {
forcedExecutionMojos.add(cacheCandidate);
} else {
if (!reconciliationExecutionMojos.isEmpty()) {
Copy link
Contributor

@AlexanderAshitkin AlexanderAshitkin Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this check for?

@@ -267,6 +286,12 @@ private CacheRestorationStatus restoreProject(
// mojoExecutionScope.seed(
// org.apache.maven.api.MojoExecution.class, new DefaultMojoExecution(cacheCandidate));
mojoExecutionRunner.run(cacheCandidate);
} else if (reconciliationExecutionMojos.contains(cacheCandidate)) {
Copy link
Contributor

@AlexanderAshitkin AlexanderAshitkin Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced that this logic is valid. If you have build 1 with the plugins [(X, parameters PX1), (Y, parameters PX1)], with the flag "on",the new cached build might be a result of [(X, parameters PX2), (Y, parameters PY1)]. The issue here is that without tracking inputs/outputs, we can't guarantee that reusing PY1 is valid if the first plugin changed parameters. It's possible that it relies on outputs from the first plugin and could yield a different result without the cache. This logic seems flawed to me at the moment. It might lead to all sorts of corruptions inadvertently. it probably could run for certain plugins in presence of additional metadata, but not in a general case.

This optimization could work for a leaf plugins (plugins which do not contribute and do not impact subsequent plugins) but that will require additional plugin metadata

ArtifactRestorationReport restorationReport = cacheController.restoreProjectArtifacts(cacheResult);
ArtifactRestorationReport restorationReport = cacheController.restoreProjectArtifacts(
cacheResult,
!containsExecution(plannedExecutions, "org.apache.maven.plugins", "maven-jar-plugin", "jar"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this hardcoded list of plugins doesnt look good and manifests a workaround which might not work in general case. What is it for?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants