-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rspec-queue does not fail if there are NameErrors in specs #100
Comments
So the answer is a bit of a yes and no. yes: We do want to exit 0 if the worker crash on boot because of a transient issue (e.g. it fails to connect to the queue, or the machine it runs on is somehow unhealthy). Hence the behavior you noticed. As for the no: Ideally this specific case should lead to an All that being said, in theory it shouldn't be too much of a problem as long as you run the |
Thanks for taking the time to explain @casperisfine, much appreciated! Until reading this I hadn't been running Though I see now its not needed with #!/usr/bin/env sh
# Perform a dry run first so all spec files are loaded but are not executed. rspec will exit with
# failure on any syntax etc. errors in the spec files.
bin/rspec --dry-run --format progress &&
bundle exec rspec-queue --queue $REDIS_URL --timeout 300 &&
bundle exec rspec-queue --queue $REDIS_URL --timeout 300 --report # Wait for test queue to be exhausted then print report.
|
I've just encountered the following behaviour with the rspec runner of ci-queue:
NameError
, e.g. by referring to a non-existent class:I guessed wrongly that
rspec-queue
would fail with a non-zero exit code, but it exited with0
.Bare
rspec
exits with1
in the above situation.I'm wondering if this is intended behaviour - perhaps its needed for retries to work?
The error message output is:
Debugging shows that line 400 is where the
0
exit code is returned, because theNameError
causes@world.non_example_failure
to betrue
. Syntax errors appear to have the same result:ci-queue/ruby/lib/rspec/queue.rb
Lines 387 to 401 in cba2cdd
The text was updated successfully, but these errors were encountered: