Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test #23

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix flaky test #23

wants to merge 2 commits into from

Conversation

ThugJudy
Copy link

This PR aims to fix the following 3 flaky test cases:

com.github.roroche.plantuml.BuildClassDiagramTaskTest.produceTests
com.github.roroche.plantuml.diagrams.ClassDiagramTest.produceTests
com.github.roroche.plantuml.diagrams.PrintDiagramTest.produceTests

I found and confirmed the flaky behavior using an open-source research tool NonDex, which shuffles implementations of nondeterminism operations.

Problem:

The order of the attributes of class that is obtained by ClassDiagramBuilder's build function is prone to changes and causes the flakiness

Solution:

Since we do not have access to the internal library, I have a added custom function that verifies that the strings are similar

Steps to reproduce
The following command can be used to reproduce the assertion errors and verify the fix.

Integrate NonDex:

Add the following snippet to the top of the build.gradle in $PROJ_DIR:

plugins {
id 'edu.illinois.nondex' version '2.1.1-1'
}
Append to build.gradle in $PROJ_DIR:

apply plugin: 'edu.illinois.nondex'
Execute Test with Gradle:

./gradlew --info test --tests com.github.roroche.plantuml.diagrams.PrintDiagramTest.produceTests
./gradlew --info test --tests com.github.roroche.plantuml.diagrams.ClassDiagramTest.produceTestsorg.apereo.portal.events.tincan.json.LrsDataModelSerializeTest.testLrsObjectSerialize
./gradlew --info test --tests com.github.roroche.plantuml.BuildClassDiagramTaskTest.produceTestsorg.apereo.portal.events.AnalyticsIncorporationComponentEventSerializationTest.testEventFiltering

Run NonDex:

./gradlew --info nondexTest --tests=com.github.roroche.plantuml.diagrams.PrintDiagramTest.produceTests --nondexRuns=50
./gradlew --info nondexTest --tests=com.github.roroche.plantuml.diagrams.ClassDiagramTest.produceTests --nondexRuns=50
./gradlew --info nondexTest --tests=com.github.roroche.plantuml.BuildClassDiagramTaskTest.produceTests --nondexRuns=50
Test Environment:

Java version "1.8.0_381"
macOS Venture Version 13.4.1 (22F82)
Please let me know if you have any concerns or questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant