Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Java parameterized test cases #302

Merged
merged 1 commit into from
Sep 16, 2024

Conversation

zabetak
Copy link
Contributor

@zabetak zabetak commented Sep 10, 2024

Add new testmode, ie., javaParamTestCase that allows to split and run in parallel Java parameterized tests. The JavaTestCaseName mode is retained for backwards compatibility reasons.

From maven-surefire-plugin:3.0.0-M6 onwards method filtering (thus parameters) are supported in includesFile and excludesFile. Some basic examples of using Surefire with parameterized tests can be found in https://github.com/zabetak/surefire-parameterized-tests

Relates to: #279, #283

Testing done

New unit test added.

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests - that demonstrates feature works or fixes the issue

From maven-surefire-plugin:3.0.0-M6 onwards method filtering (thus parameters) are supported in includesFile and excludesFile.

Add new testmode that allows to split and run in parallel Java parameterized tests. The JavaTestCaseName mode is retained for backwards compatibility reasons.
@zabetak zabetak requested a review from a team as a code owner September 10, 2024 09:19
@Vlatombe
Copy link
Member

The JavaTestCaseName mode is retained for backwards compatibility reasons.

Do we have to? method filtering is already required for this implementation, so do you need anything more recent than that to use the new mode?

@zabetak
Copy link
Contributor Author

zabetak commented Sep 10, 2024

@Vlatombe For my use-case, I want to be able to split parameterized tests into multiple buckets but I was thinking that some users may not so that's why I kept the previous mode. If we always want to consider parameters then we are mostly talking about a revert of #283

@Vlatombe
Copy link
Member

I reverted it at the time because with the scenarios I tried it with, surefire was not working.
I tried your example, with
https://github.com/zabetak/surefire-parameterized-tests/blob/39c0445864809acd3225d63ab531cd55d4f360e4/src/test/java/com/github/zabetak/TestClass2.java#L30-L32, works fine.

However, I tried with https://github.com/jenkinsci/kubernetes-plugin/blob/9c1d58d9badd988622b4772632037126176e1ef0/src/test/java/org/csanchez/jenkins/plugins/kubernetes/casc/EnvVarCasCTest.java#L29-L35 and it ignores parameters.

Seems to me that surefire has imperfect support for method filtering with parameters.

@zabetak
Copy link
Contributor Author

zabetak commented Sep 10, 2024

I added another test case in zabetak/surefire-parameterized-tests@eead295 to simulate the EnvVarCasCTest.java which has multiple parameters but I don't see an issue with surefire.

There may be a bug in Surefire but it may not be that common after all. On the other hand if we retain the old mode and just add a new one there is no risk for regressions and we can still split based on params for those users that need this feature.

@Vlatombe
Copy link
Member

Vlatombe commented Sep 16, 2024

I spent some time trying to understand what was the difference between sample project and the real project. I guess it is a subtle surefire bug, but I can't figure it out.

@Vlatombe Vlatombe merged commit 751ab24 into jenkinsci:master Sep 16, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants