You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a test plan that creates a GPG key in before. I had upgraded GPG since the last time I'd run GPG, so the running gpg-agent didn't match /bin/gpg. This caused an error whenever it tried to run GPG. That is to say, I have a before function that started erroring for reasons unrelated to my test suite or the code being tested.
roundup handled this error in before by aborting, pretending that any pending tests or test plans did not exist.
This meant that the test plan with the failing before showed up as having 0 tests. I, being an alert developer, knew that I had no test plans with 0 tests, and that roundup must be wrong.
Upon hacking roundup to deal with this in a better way, I discovered that I had several other tests that had been failing for much longer; I had about 20 tests that roundup had silently been skipping for many months (as a different before/after had been failing)!
This is a critical bug for a testing framework. Title is only a slight exaggeration (rm -rf on somewhere important would be worse).
The text was updated successfully, but these errors were encountered:
LukeShu
added a commit
to LukeShu/roundup
that referenced
this issue
Apr 11, 2017
See the comments for a fuller explanation of the changes.
This should obsolete bmizerany#23 (though
the test cases from it should still be merged).
This should fixbmizerany#29 .
This should fixbmizerany#44 .
See the comments for a fuller explanation of the changes.
This should obsolete bmizerany#23 (though
the test cases from it should still be merged).
This should fixbmizerany#29 .
This should fixbmizerany#44 .
I have a test plan that creates a GPG key in
before
. I had upgraded GPG since the last time I'd run GPG, so the runninggpg-agent
didn't match/bin/gpg
. This caused an error whenever it tried to run GPG. That is to say, I have abefore
function that started erroring for reasons unrelated to my test suite or the code being tested.roundup handled this error in
before
by aborting, pretending that any pending tests or test plans did not exist.This meant that the test plan with the failing
before
showed up as having 0 tests. I, being an alert developer, knew that I had no test plans with 0 tests, and that roundup must be wrong.Upon hacking roundup to deal with this in a better way, I discovered that I had several other tests that had been failing for much longer; I had about 20 tests that roundup had silently been skipping for many months (as a different before/after had been failing)!
This is a critical bug for a testing framework. Title is only a slight exaggeration (
rm -rf
on somewhere important would be worse).The text was updated successfully, but these errors were encountered: