unbork our coverage setup #141
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
flbraun
force-pushed
the
unbork-coverage
branch
18 times, most recently
from
79ccb43 to
1f30bf1
Compare
flbraun
changed the title
flbraun
force-pushed
the
unbork-coverage
branch
3 times, most recently
from
4b30aba to
9aae9cc
Compare
flbraun
force-pushed
the
unbork-coverage
branch
7 times, most recently
from
3224bd8 to
85d691f
Compare
flbraun
force-pushed
the
unbork-coverage
branch
from
85d691f to
cbd4885
Compare
there are a couple of problems with our coverage setup: 1. the tox envs generate and attempt to merge coverage files, leading to race conditions. 2. after tox has run, the `tox` make target and the `test` workflow both _also_ try to process the coverage data. 3. the `test` workflow uploads a report to Codecov for each permutation of the python-django-matrix. this is not only inefficient, but also distorts our coverage ratings. at Codecov we're currently scoring 91% coverage, although it should be 93%. this is caused by not properly merging the coverage results of the test matrix and instead uploading them individually, causing our coverage score to be that of the last permutation to finish. 4. for some unexplained reason, usage of the `codecov/codecov-action` was replaced with Codecov's bash importer in fbd0cb1. at that point in time the bash importer had already been deprecated for 18 months (!) and the related repository had already been archived for 8 months. users are officially encouraged to migrate to a newer importer (as implemented in the previously used action). problems 1-3 are addressed by implementing separation of concerns: test runs (whether in workflows or from make targets) now only emit partial coverage data files. combining them and generating reports is done afterward. to address problem 4 the previously used `codecov/codecov-action` action is reinstated and properly configured. finally, re-enable parallel tox runs for the `tox` make target. this was temporarily disabled to circumvent problems 1 and 2 in 7e92f08.
flbraun
force-pushed
the
unbork-coverage
branch
from
cbd4885 to
fb6b260
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
there are a couple of problems with our coverage setup:
toxmake target and thetestworkflow both also try to process the coverage data.testworkflow uploads a report to Codecov for each permutation of the python-django-matrix. this is not only inefficient, but also distorts our coverage ratings. at Codecov we're currently scoring 91% coverage, although it should be 93%. this is caused by not properly merging the coverage results of the test matrix and instead uploading them individually, causing our coverage score to be that of the last permutation to finish.codecov/codecov-actionwas replaced with Codecov's bash importer in fbd0cb1. at that point in time the bash importer had already been deprecated for 18 months (!) and the related repository had already been archived for 8 months. users are officially encouraged to migrate to a newer importer (as implemented in the previously used action).problems 1-3 are addressed by implementing separation of concerns: test runs (whether in workflows or from make targets) now only emit partial coverage data files. combining them and generating reports is done afterward.
to address problem 4 the previously used
codecov/codecov-actionaction is reinstated and properly configured.finally, re-enable parallel tox runs for the
toxmake target. this was temporarily disabled to circumvent problems 1 and 2 in 7e92f08.