Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core: Race condition in result handler #919

Open
nik-redhat opened this issue Aug 3, 2021 · 0 comments
Open

Core: Race condition in result handler #919

nik-redhat opened this issue Aug 3, 2021 · 0 comments
Assignees
Labels
Core Change in core components Critical This is a critical issue and needs to be handled properly with test case support and design thought.
Milestone

Comments

@nik-redhat
Copy link
Collaborator

nik-redhat commented Aug 3, 2021

Describe the bug
The results being logged into the terminal by the result handler, sometimes misses out on the result of the last run even though the test ran completely. For example:

✔ Param Handling Success.
✔ Test List built
✔ Log dir creation successful.
✔ Environment setup successful.
✔ Running test case : test_arb_to_repl_conversion_with_io-arb
✖ test_arb_to_repl_conversion_with_io-arb Failed
✔ Running test case : test_arb_to_repl_conversion_with_io-dist-arb
✖ test_arb_to_repl_conversion_with_io-dist-arb Failed
Results for afr
Test nature : disruptive
TestName : test_arb_to_repl_conversion_with_io
+-------------+-------------+-----------------------+-------------+
| Volume Type | Test Result | Time Taken (hh:mm:ss) | Skip Reason |
+-------------+-------------+-----------------------+-------------+
|     arb     |     FAIL    |        00:29:14       |      NA     |
+-------------+-------------+-----------------------+-------------+
+--------------------------------------+-------+
|               Category               | Count |
+--------------------------------------+-------+
|           Disruptive Tests           |   1   |
|         Non Disruptive Tests         |   0   |
|       Disruptive Tests Skipped       |   0   |
|     Non Disruptive Tests Skipped     |   0   |
|              Tests Ran               |   1   |
|            Tests Skipped             |   0   |
|             Total Tests              |   1   |
|   Disruptive Tests Pass Percentage   |  0.0  |
| Non Disruptive Tests Pass Percentage |   0   |
|        Total Pass Percentage         |  0.0  |
+--------------------------------------+-------+
Total Time Taken : 00 hours 58 minutes 57 seconds
✔ Tearing down successful.

In the above test run, the second test dist-arb result was not present in the table even though the framework ran that (which is confirmed from the logs and the time taken). Though, the results are logged sometime like:

✔ Param Handling Success.
✔ Test List built
✔ Log dir creation successful.
✔ Environment setup successful.
✔ Running test case : test_arb_to_repl_conversion_with_io-arb
✖ test_arb_to_repl_conversion_with_io-arb Failed
✔ Running test case : test_arb_to_repl_conversion_with_io-dist-arb
✖ test_arb_to_repl_conversion_with_io-dist-arb Failed
Results for afr
Test nature : disruptive
TestName : test_arb_to_repl_conversion_with_io
+-------------+-------------+-----------------------+-------------+
| Volume Type | Test Result | Time Taken (hh:mm:ss) | Skip Reason |
+-------------+-------------+-----------------------+-------------+
|     arb     |     FAIL    |        00:09:32       |      NA     |
|   dist-arb  |     FAIL    |        00:10:17       |      NA     |
+-------------+-------------+-----------------------+-------------+
+--------------------------------------+-------+
|               Category               | Count |
+--------------------------------------+-------+
|           Disruptive Tests           |   1   |
|         Non Disruptive Tests         |   0   |
|       Disruptive Tests Skipped       |   0   |
|     Non Disruptive Tests Skipped     |   0   |
|              Tests Ran               |   1   |
|            Tests Skipped             |   0   |
|             Total Tests              |   1   |
|   Disruptive Tests Pass Percentage   |  0.0  |
| Non Disruptive Tests Pass Percentage |   0   |
|        Total Pass Percentage         |  0.0  |
+--------------------------------------+-------+
Total Time Taken : 00 hours 20 minutes 01 seconds
✔ Tearing down successful.

It is being suspected that this issue is occurring due to a race condition in the multiprocessing queue..

@nik-redhat nik-redhat added Core Change in core components Critical This is a critical issue and needs to be handled properly with test case support and design thought. labels Aug 3, 2021
@nik-redhat nik-redhat added this to the Iter 7 milestone Aug 3, 2021
@schaffung schaffung self-assigned this Aug 10, 2021
@nik-redhat nik-redhat pinned this issue Aug 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Core Change in core components Critical This is a critical issue and needs to be handled properly with test case support and design thought.
Projects
None yet
Development

No branches or pull requests

2 participants