Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate "Difference" Report Between Two TestResult.xml files. #13

Open
TylerNielsen opened this issue Jul 18, 2018 · 2 comments
Open

Comments

@TylerNielsen
Copy link

TylerNielsen commented Jul 18, 2018

Summary
Create an html report that shows the differences between two NUnit runs (New tests, removed tests, tests now failing, tests now passing).

Detailed
In short, I want to be able to have my results from Master, run the tests on my feature branch, then see which tests I fixed and/or broke as a result. There are many integrations/tools out there to help visualize results from NUnit tests. The Test Results view within Jenkins provides a great Diff report, however it only works for two concurrent builds of the same job. If I want to take an build from a separate branch and compare to the latest build of master, I'm stuck trying to do XML diffs.

I've scoured the internet as I would think I'm not the only one looking to do something like this, but I've come up empty. Please let me know if you can think of any other solutions to this problem.

It seems this project hasn't been touched in a while, this may not be the right place for this, but it's the closest I've seen to a generic NUnit solution. I'd be willing to help work on this if you think it's worthwhile.

Version
I'm currently working with NUnit 3.9, so it looks like this project would need to first support the new TestResult.xml formats (from #11).

@CharliePoole
Copy link
Owner

@TylerNielsen You're right - I haven't worked on this for a long time and it basically supports NUnit V2. I'd be glad to take a PR on it, first for #11 and subsequently for this one.

I do think that for NUnit 3, the best approach for a single-result report is to create a test engine extension. Certainly, such an extension would be easy to develop as a repackaging of this program, once NUnit 3 is supported. For comparisons, I think a stand-alone program like this is probably the best option.

@TylerNielsen
Copy link
Author

For #11 to be closed, would it need to support each incremental version up to the current, or could we just skip straight to 3.9 format? (I haven't yet looked to see if there are actually incremental changes to the .xml format after the jump from 2.X to 3.X).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants