Use pytest-testmon to run the long integration tests that are affected by the open PR

[xzdandy]

Search before asking

• I have searched the EvaDB issues and found no similar feature requests.

Description

For now, if open's pr adds or changes long integration test, they are not tested by circle ci before merging into staging branch. We can use pytest --testmon and circleci to cache the .testmondata, to only run the long integration tests are affected by the PR.

Hi @jiashenC, what do you think? Thanks!

Use case

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[jiashenC]

Sure, this should be very useful.

[jiashenC]

@xzdandy I had a chance to test this, but it seems not working very well. Most tests need to be rerun even though I didn't change any source code. You can try running tests again using my PR.

[xzdandy]

@jiashenC Thanks for trying. I will give a try. I think testmon is using coverage, so we can not do no:cov.

[xzdandy]

We need to use python -m pytest --testmon-forceselect test/integration_tests/long/ -m "not benchmark".

With --testmon, the -m option will overwrite the testmon's selection. I will update the #1209 accordingly.

Reference: https://testmon.org/#all-command-line-options