Improve Smart Testing Performance

[dipak-pawar]

• Run Smart-Testing with any real project.
• Find all areas which impacts execution time/performance using profiling.
• After finding all areas, find optimized solutions for it.

[MatousJobanek]

I have one idea how to improve the performance:
The calculations (scm) would be executed in a separated thread, so the main thread wouldn't be blocked and could continue with the Maven build. I guess that calculation should be probably finished before the surefire execution starts (in most of the cases), but this has to be managed anyway (in cases when it takes more time than the compilation etc...).
To avoid the state when the surefire starts before the calculation is finished and tries to read non-existing or uncomplete scm-changes file, the logic should use some marker. This marker can be a temporary empty file that would represent that the execution is still being processed. When the calculation execution is finished, the file is removed. On the surefire side, this file would be checked if whether still exists or not. If not, then it can start looking/reading the scm-changes file and continue with normal test execution.

[lordofthejars]

Yes it is a good idea, even we could create some kind of chain on this. For example when scm changes has been calculated, if affected strategy is enabled, start calculating graph too. But maybe this is something to take into consideration as next step.

[MatousJobanek]

@lordofthejars IIRC you have some big project with a lot of test classes. Is it AssertJ or something else?

[MatousJobanek]

if affected strategy is enabled

If we don't want to detect which strategies are activated in the maven extension phase, then we cannot create this kind of chain.
Instead of chaining the strategy specific tasks, we could introduce another level of optimization - running the activated strategy implementations in parallel.

[lordofthejars]

Yes it was just assertj

[bartoszmajsak]

@lordofthejars can you share modified project so we can have a simple reference to reproducer?

[lordofthejars]

There were no modification, I just cloned the assertj project and modified org.assertj.core.util.Strings_join_Test with one redundant test

[bartoszmajsak]

And how did you execute the build? Which version of ST was defined in extensions.xml?

[lordofthejars]

In the git history of workshop you can see next things if you want to reproduce exactly the same thing:

• Affected strategy: lordofthejars/smart-testing-workshop@f36cf84

You need to run this using affected strategy and it is where the 2 million nodes are created.

• Changed: lordofthejars/smart-testing-workshop@951b7c5

You need to run this using changed strategy, but no performance issues observed at that time.

[bartoszmajsak]

Thanks, that's exactly the context we will need when we dive into this analysis!