Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spurious fails with too many (>10) tests #206

Closed
ghost opened this issue May 18, 2017 · 1 comment
Closed

Spurious fails with too many (>10) tests #206

ghost opened this issue May 18, 2017 · 1 comment

Comments

@ghost
Copy link

ghost commented May 18, 2017

Summary

Each of my tests passes if I run it on its own.
If I run them all, my first 10 tests pass, all after that fail.
If I skip the first few, I still get 10 passes.

Detail

I have a GUI and I'm building suite of AssertJ tests, 19 so far.
I have tried to make all my tests completely independent of one another, but there seems to be some hidden dependency that I'm unaware of.
Usually, I've been running only the test I'm working on, and skipping the others with @ignore.
Sometimes I run them all, just for completeness, but since I got past 10 tests I started getting spurious fails. I now have 19 tests.

By a process of selecting and deselecting tests, I find that the first 10 tests in any run will pass, and all tests after that will fail.
If I try to run all 19 tests, numbers 1-10 pass and numbers 11-19 fail.
If I @ignore #1, numbers 2-10 still pass, and so does #11 (which failed before). #12 onwards still fail.
If I @ignore #1-#4, #5-#14 pass and #15-#19 fail.
If I @ignore 1-9, 10-19 all pass.
Whatever I do in any run, the first ten tests in that run pass, and any after that fail.

In each test, my GUI program is launched, opens a file, then does various things with it depending on the content of the test. When a test fails, it always fails with a timeout at the file loading stage.

I'm mystified. Has any had this kind of problem before?

@ghost
Copy link
Author

ghost commented Jun 2, 2017

Eventually traced this to the use of a SwingWorker to handle non-EDT tasks in my GUI.
I don't think that's a bad thing in itself, but it doesn't play well with AssertJ-Swing.

When assertj-swing runs a GUI multiple times, it seems that each instance of the GUI uses up a thread from the SwingWorker thread pool, and (try as I might) doesn't release it on closing.
The default SwingWorker thread pool size is 10 - exactly the same as the number of tests I can successfully run.

I eventually fixed this with a one-line change. In my GUI, where I start the SwingWorker, I replaced

worker.execute();

with

new Thread(worker).start();

Problem solved.

@ghost ghost closed this as completed Jun 2, 2017
@ghost ghost changed the title Spurious fails with too many tests Spurious fails with too many (>10) tests Jun 2, 2017
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants