Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test history refactoring and improvements #625

Merged
merged 85 commits into from
Jul 24, 2024
Merged
Show file tree
Hide file tree
Changes from 73 commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
3dfeb90
Allow disabling test name mangling (keepTestNames).
May 12, 2022
589bcc1
Fix tests.
May 12, 2022
5deff15
Improve comment.
May 30, 2022
84367f8
Assume history always available.
Sep 21, 2022
b11b825
Paralellize test history handling to unbreak it with large amounts of…
Sep 21, 2022
c29f0cc
Implement explicit XML parsing to avoid slow reflection access.
Sep 26, 2022
379ef56
Make test history dynamic and increase table size.
Sep 26, 2022
8ba78e7
Fix trend charts not shown after carousel slide.
Sep 27, 2022
0bba408
Fix carousel not redrawing.
Sep 27, 2022
165207c
Show same range as table in test result trend charts.
Sep 28, 2022
3cb7ab3
Show test status in duration chart.
Sep 28, 2022
e2d7b54
Remove the other test history chart.
Sep 28, 2022
6a7f600
Support 'count' parameter in history URLs.
Sep 29, 2022
7f2a9e7
Improve test history appearance.
Sep 29, 2022
673397a
Improve test history appearance.
Sep 29, 2022
60d9ef8
Remove carousel as charts are no longer async (and because it was a b…
Sep 30, 2022
fae3c33
Make it compatible to XUnit plugin.
Oct 4, 2022
d195628
Improve test history appearance.
Oct 4, 2022
0142b86
Put both charts into one.
Oct 5, 2022
d68d1c9
Support Dark Reader to some extent.
Oct 6, 2022
8a682dc
Improve test history appearance.
Oct 7, 2022
21e7a9c
Improve chart appearance.
Oct 10, 2022
80f2727
Fix history URL.
Oct 10, 2022
03f950b
Implement cache for TestResult.
Oct 10, 2022
955443b
Less sorting on test result freeze.
Oct 10, 2022
65db008
Improve test history chart appearance.
Oct 12, 2022
ad7ec09
Update POM.
Oct 12, 2022
81205f6
Fix JUnit test result writes not updated in cache.
Oct 21, 2022
72589da
Update POM.
Oct 21, 2022
81c21f5
Fix build.
Mar 3, 2023
bbb187c
Fix test.
Mar 3, 2023
9230862
Fix test.
Mar 3, 2023
f04f1b6
Fix test history alignment.
Mar 6, 2023
88c6771
Clamp test case duration between 0 and 1 weeks.
Jul 9, 2023
d246a04
Clamp test case duration between 0 and reasonable limits for cases an…
Jul 9, 2023
a692bbb
Merge branch 'keep-testnames-1177' of https://github.com/mdealer/juni…
Jun 25, 2024
a6cc2a4
Fix history charts.
Jul 9, 2024
747c9a7
Less logging.
Jul 9, 2024
84fbe3f
Fix doc comments.
Jul 10, 2024
f3d05bd
Fix spotBugs warnings.
Jul 10, 2024
110f12d
Fix spotBugs warnings.
Jul 10, 2024
28fb187
Unbreak test.
Jul 10, 2024
05b3346
Unbreak test.
Jul 10, 2024
eeea84b
Update src/main/java/hudson/tasks/junit/CaseResult.java
mdealer Jul 11, 2024
b58eedc
Revert version and convert tab to space in POM.
Jul 11, 2024
dddb963
Apply suggestions from code review
mdealer Jul 11, 2024
6a2808b
Remove commented code.
Jul 11, 2024
52df2ee
Merge branch 'test-history-refactor-1265' of https://github.com/mdeal…
Jul 11, 2024
03b23bd
Remove unused code.
Jul 11, 2024
c67d580
Update based on code review.
Jul 11, 2024
3b9b1da
Update based on code review.
Jul 11, 2024
dee1923
Fix build.
Jul 11, 2024
0f7a91a
Remove commented code and add an actual comment.
Jul 11, 2024
2d873b2
Remove commented code.
Jul 11, 2024
553a073
Improve file name when saving image.
Jul 11, 2024
6190782
Shorten duration axes values.
Jul 11, 2024
d7df185
Shorten trend, smooth graph values and duration distribution axes val…
Jul 12, 2024
8e41d6b
Use Jenkins styles and move elements slightly.
Jul 12, 2024
c28ea2b
Switch to linear regression comp from ssj.
Jul 12, 2024
e19a7e9
Fix alignment.
Jul 12, 2024
1e73c69
Tabs to spaces.
Jul 12, 2024
43e0f04
Replace XMLEventReader with XMLStreamReader.
Jul 16, 2024
9ed5c0d
Update parallel-collectors to 2.6.1.
Jul 16, 2024
21b8a10
Add necessary code from ca.umontreal.iro.simul instead of import, red…
Jul 16, 2024
8221ed1
Less math doc comments.
Jul 16, 2024
173f4ad
Less math doc comments.
Jul 16, 2024
eec4876
Less math doc comments and less unused code.
Jul 16, 2024
0b39f76
Less math doc comments and less unused code.
Jul 16, 2024
b242758
Less math doc comments and less unused code.
Jul 16, 2024
fe58ec4
Improve safe XML factory handling.
Jul 17, 2024
9c67afe
Improve safe XML factory handling.
Jul 17, 2024
2554629
Add missing check for EXTRA_GRAPH_MATH_ENABLED.
Jul 17, 2024
7247f43
Apply some colors from CSS to history charts.
Jul 17, 2024
610a648
Switch back to ssj dependency but exclude gson due to CVEs.
Jul 18, 2024
331b009
Switch build window size links to a drop down box.
Jul 18, 2024
d293576
Switch from currentTimeMillis to nanoTime where appropriate.
Jul 18, 2024
682a161
Improve history page CSP compliance, clean up JavaScript side.
Jul 19, 2024
1df4241
Implement sample size to speed up navigating huge results greatly.
Jul 19, 2024
2764ca3
Hide smooth and trend graphs on narrow screens.
Jul 22, 2024
f22740f
Improve backward compatibility.
Jul 23, 2024
45afd2f
Merge branch 'master' into test-history-refactor-1265
timja Jul 23, 2024
4ca39a7
Merge branch 'test-history-refactor-1265' of github.com:mdealer/junit…
timja Jul 23, 2024
6cb42d1
Fix properties not loaded.
Jul 23, 2024
d958cd6
Merge branch 'test-history-refactor-1265' of https://github.com/mdeal…
Jul 23, 2024
31b862c
Implement XML parsing for more fields (including properties).
Jul 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -189,11 +189,21 @@
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>jackson2-api</artifactId>
</dependency>
<dependency>
<groupId>com.pivovarit</groupId>
<artifactId>parallel-collectors</artifactId>
<version>2.6.1</version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is up-to-date, but Renovate is trying to upgrade it to version 3.x in e.g. #629, which won't work because 3.x requires Java 21 or newer. Can we update the Renovate configuration to exclude 3.x until we are ready to require Java 21 or newer? That way, we will continue to get updates to version 2.x.

</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>pipeline-utility-steps</artifactId>
<scope>test</scope>
</dependency>
<dependency>
mdealer marked this conversation as resolved.
Show resolved Hide resolved
<groupId>colt</groupId>
<artifactId>colt</artifactId>
<version>1.2.0</version>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
Expand Down
131 changes: 112 additions & 19 deletions src/main/java/hudson/tasks/junit/CaseResult.java
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,15 @@
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;

import static java.util.Collections.emptyList;
import static java.util.Collections.singletonList;

import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamReader;

/**
* One test result.
*
Expand All @@ -65,7 +69,7 @@
*/
public class CaseResult extends TestResult implements Comparable<CaseResult> {
private static final Logger LOGGER = Logger.getLogger(CaseResult.class.getName());
private final float duration;
private float duration;
/**
* Start time in epoch milliseconds - default is -1 for unset
*/
Expand All @@ -74,16 +78,17 @@ public class CaseResult extends TestResult implements Comparable<CaseResult> {
* In JUnit, a test is a method of a class. This field holds the fully qualified class name
* that the test was in.
*/
private final String className;
private String className;
/**
* This field retains the method name.
*/
private final String testName;
private String testName;
private transient String safeName;
private final boolean skipped;
private final String skippedMessage;
private final String errorStackTrace;
private final String errorDetails;
private boolean skipped;
private boolean keepTestNames;
private String skippedMessage;
private String errorStackTrace;
private String errorDetails;
private final Map<String, String> properties;
@SuppressFBWarnings(value = "SE_TRANSIENT_FIELD_NOT_RESTORED", justification = "Specific method to restore it")
private transient SuiteResult parent;
Expand All @@ -98,14 +103,14 @@ public class CaseResult extends TestResult implements Comparable<CaseResult> {
* If these information are reported at the test case level, these fields are set,
* otherwise null, in which case {@link SuiteResult#stdout}.
*/
private final String stdout,stderr;
private String stdout,stderr;

/**
* This test has been failing since this build number (not id.)
*
* If {@link #isPassed() passing}, this field is left unused to 0.
*/
private /*final*/ int failedSince;
private int failedSince;

private static float parseTime(Element testCase) {
String time = testCase.attributeValue("time");
Expand Down Expand Up @@ -138,6 +143,7 @@ public CaseResult(SuiteResult parent, String testName, String errorStackTrace, S
this.skipped = false;
this.skippedMessage = null;
this.properties = Collections.emptyMap();
this.keepTestNames = false;
}

@Restricted(Beta.class)
Expand Down Expand Up @@ -165,14 +171,15 @@ public CaseResult(
this.skipped = skippedMessage != null;
this.skippedMessage = skippedMessage;
this.properties = Collections.emptyMap();
this.keepTestNames = false;
}

@Deprecated
CaseResult(SuiteResult parent, Element testCase, String testClassName, boolean keepLongStdio, boolean keepProperties) {
this(parent, testCase, testClassName, StdioRetention.fromKeepLongStdio(keepLongStdio), keepProperties);
CaseResult(SuiteResult parent, Element testCase, String testClassName, boolean keepLongStdio, boolean keepProperties, boolean keepTestNames) {
this(parent, testCase, testClassName, StdioRetention.fromKeepLongStdio(keepLongStdio), keepProperties, keepTestNames);
}

CaseResult(SuiteResult parent, Element testCase, String testClassName, StdioRetention stdioRetention, boolean keepProperties) {
CaseResult(SuiteResult parent, Element testCase, String testClassName, StdioRetention stdioRetention, boolean keepProperties, boolean keepTestNames) {
// schema for JUnit report XML format is not available in Ant,
// so I don't know for sure what means what.
// reports in http://www.nabble.com/difference-in-junit-publisher-and-ant-junitreport-tf4308604.html#a12265700
Expand Down Expand Up @@ -200,7 +207,7 @@ public CaseResult(
errorStackTrace = getError(testCase);
errorDetails = getErrorMessage(testCase);
this.parent = parent;
duration = parseTime(testCase);
duration = clampDuration(parseTime(testCase));
this.startTime = -1;
skipped = isMarkedAsSkipped(testCase);
skippedMessage = getSkippedMessage(testCase);
Expand All @@ -226,6 +233,80 @@ public CaseResult(
}
}
this.properties = properties;
this.keepTestNames = keepTestNames;
}

public CaseResult(CaseResult src) {
this.duration = src.duration;
this.className = src.className;
this.testName = src.testName;
this.skippedMessage = src.skippedMessage;
this.skipped = src.skipped;
this.keepTestNames = src.keepTestNames;
this.errorStackTrace = src.errorStackTrace;
this.errorDetails = src.errorDetails;
this.failedSince = src.failedSince;
this.stdout = src.stdout;
this.stderr = src.stderr;
this.properties = new HashMap<>();
this.properties.putAll(src.properties);
}

public static float clampDuration(float d) {
return Math.min(365.0f * 24 * 60 * 60, Math.max(0.0f, d));
}

public static CaseResult parse(SuiteResult parent, final XMLStreamReader reader, String ver) throws XMLStreamException {
CaseResult r = new CaseResult(parent, null, null, null);
while (reader.hasNext()) {
final int event = reader.next();
if (event == XMLStreamReader.END_ELEMENT && reader.getLocalName().equals("case")) {
return r;
}
if (event == XMLStreamReader.START_ELEMENT) {
final String elementName = reader.getLocalName();
switch (elementName) {
case "duration":
r.duration = clampDuration(new TimeToFloat(reader.getElementText()).parse());
break;
case "className":
r.className = reader.getElementText();
break;
case "testName":
r.testName = reader.getElementText();
break;
case "skippedMessage":
r.skippedMessage = reader.getElementText();
break;
case "skipped":
r.skipped = Boolean.parseBoolean(reader.getElementText());
break;
case "keepTestNames":
r.keepTestNames = Boolean.parseBoolean(reader.getElementText());
break;
case "errorStackTrace":
r.errorStackTrace = reader.getElementText();
break;
case "errorDetails":
r.errorDetails = reader.getElementText();
break;
case "failedSince":
r.failedSince = Integer.parseInt(reader.getElementText());
break;
case "stdout":
r.stdout = reader.getElementText();
break;
case "stderr":
r.stderr = reader.getElementText();
break;
default:
if (LOGGER.isLoggable(Level.FINEST)) {
LOGGER.finest("CaseResult.parse encountered an unknown field: " + elementName);
}
}
}
}
return r;
}

static String possiblyTrimStdio(Collection<CaseResult> results, StdioRetention stdioRetention, String stdio) { // HUDSON-6516
Expand Down Expand Up @@ -354,7 +435,7 @@ public String getDisplayName() {
private String getNameWithEnclosingBlocks(String rawName) {
// Only prepend the enclosing flow node names if there are any and the run this is in has multiple blocks directly containing
// test results.
if (!getEnclosingFlowNodeNames().isEmpty()) {
if (!keepTestNames && !getEnclosingFlowNodeNames().isEmpty()) {
Run<?, ?> r = getRun();
if (r != null) {
TestResultAction action = r.getAction(TestResultAction.class);
Expand Down Expand Up @@ -579,15 +660,27 @@ public String getStderr() {
return getSuiteResult().getStderr();
}

static int PREVIOUS_TEST_RESULT_BACKTRACK_BUILDS_MAX =
Integer.parseInt(System.getProperty(History.HistoryTableResult.class.getName() + ".PREVIOUS_TEST_RESULT_BACKTRACK_BUILDS_MAX","25"));

@Override
public CaseResult getPreviousResult() {
if (parent == null) return null;

TestResult previousResult = parent.getParent().getPreviousResult();
if (previousResult == null) return null;
if (previousResult instanceof hudson.tasks.junit.TestResult) {
hudson.tasks.junit.TestResult pr = (hudson.tasks.junit.TestResult) previousResult;
return pr.getCase(parent.getName(), getTransformedFullDisplayName());
TestResult previousResult = parent.getParent();
int n = 0;
while (previousResult != null && n < PREVIOUS_TEST_RESULT_BACKTRACK_BUILDS_MAX) {
previousResult = previousResult.getPreviousResult();
if (previousResult == null)
return null;
if (previousResult instanceof hudson.tasks.junit.TestResult) {
hudson.tasks.junit.TestResult pr = (hudson.tasks.junit.TestResult) previousResult;
CaseResult cr = pr.getCase(parent.getName(), getTransformedFullDisplayName());
if (cr != null) {
return cr;
}
}
++n;
}
return null;
}
Expand Down
9 changes: 3 additions & 6 deletions src/main/java/hudson/tasks/junit/ClassResult.java
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,7 @@
import org.kohsuke.stapler.StaplerResponse;
import org.kohsuke.stapler.export.Exported;

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.*;

/**
* Cumulative test result of a test class.
Expand All @@ -44,7 +42,7 @@ public final class ClassResult extends TabulatedResult implements Comparable<Cla
private final String className; // simple name
private transient String safeName;

private final List<CaseResult> cases = new ArrayList<>();
private final Set<CaseResult> cases = new TreeSet<CaseResult>();

private int passCount,failCount,skipCount;

Expand Down Expand Up @@ -146,7 +144,7 @@ public Object getDynamic(String name, StaplerRequest req, StaplerResponse rsp) {

@Exported(name="child")
@Override
public List<CaseResult> getChildren() {
public Collection<CaseResult> getChildren() {
timja marked this conversation as resolved.
Show resolved Hide resolved
return cases;
}

Expand Down Expand Up @@ -216,7 +214,6 @@ else if(r.isPassed()) {

void freeze() {
this.tally();
Collections.sort(cases);
}

public String getClassName() {
Expand Down
Loading
Loading