Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make sure tests generated by pytest plugins are not ignored. #8425

Closed
zztalker opened this issue Nov 7, 2019 · 8 comments
Closed

Make sure tests generated by pytest plugins are not ignored. #8425

zztalker opened this issue Nov 7, 2019 · 8 comments
Assignees
@eleanorjboyd
Labels
area-testing feature-request Request for new features or functionality needs community feedback Awaiting community feedback needs PR Ready to be worked on

Comments

@zztalker
Copy link

zztalker commented Nov 7, 2019 •
edited by ericsnowcurrently
Loading

Need to improve handling pytest-plugins tests, because after resolving #7287 they are skipped.

[added by @ericsnowcurrently]:

Just to be clear, this is about pytest plugins that add in extra test nodes/items during the "collection" phase. By default pytest collects tests/suites/modules/packages on the filesystem (and optionally doctests). This is what the testing adapter script is focused on accommodating.

Additionally, pytest allows plugins to add test nodes (e.g. the pep8 plugin). However, the adapter doesn't know how to handle those. In #8242 we start ignoring them instead of failing. That isn't ideal, so we should work on supporting plugin-generated test nodes.

FWIW, I am not sure what the old code (parsing the pytest CLI output) used to do with plugin-generated nodes. My vague recollection is that it ignored them as well (but that should probably be verified). That is the reason this is an enhancement request.
@zztalker zztalker added triage-needed Needs assignment to the proper sub-team feature-request Request for new features or functionality labels Nov 7, 2019
@zztalker
Copy link
Author

zztalker commented Nov 7, 2019

@ericsnowcurrently please, take a look.

@zztalker zztalker mentioned this issue Nov 7, 2019
Merged
10 tasks
@ericsnowcurrently ericsnowcurrently added area-testing needs PR debt Covers everything internal: CI, testing, refactoring of the codebase, etc. and removed feature-request Request for new features or functionality labels Nov 7, 2019
@ghost ghost removed the triage-needed Needs assignment to the proper sub-team label Nov 7, 2019
@ericsnowcurrently
Copy link
Member

Thanks for opening this issue, @zztalker. Did I understand correctly from PR #8242 that you are going to work on a separate PR for this. If so, go for it! :) In the meantime we'll take a look as soon as we can.

@luabud

@ericsnowcurrently ericsnowcurrently added needs decision feature-request Request for new features or functionality and removed needs PR debt Covers everything internal: CI, testing, refactoring of the codebase, etc. labels Nov 7, 2019
@zztalker zztalker mentioned this issue May 15, 2020
Closed
9 tasks
wmanley added a commit to stb-tester/stbt-rig that referenced this issue Apr 2, 2021
@wmanley
pytest: Tests: Derive from pytest.Function rather than Item
0b29db3 
This way VSCode will still include us in the list of results.

See also: [vscode-python#8242] where it broke, [vscode-python#8245] which
is the issue tracking fixing it and [_get_item_kind] which is the function
which figures out whether this is an pytest test function or not.

[vscode-python#8242]: microsoft/vscode-python#8242
[vscode-python#8245]: microsoft/vscode-python#8425
[_get_item_kind]: https://github.com/microsoft/vscode-python/blob/411d1d4ae7e3884a8ef9294a0aa1688dc05d0f37/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py#L530-L540
wmanley added a commit to stb-tester/stbt-rig that referenced this issue Apr 2, 2021
@wmanley
pytest: Tests: Derive from pytest.Function rather than Item
89f7a27 
This way VSCode will still include us in the list of results.

See also: [vscode-python#8242] where it broke, [vscode-python#8245] which
is the issue tracking fixing it and [_get_item_kind] which is the function
which figures out whether this is an pytest test function or not.

[vscode-python#8242]: microsoft/vscode-python#8242
[vscode-python#8245]: microsoft/vscode-python#8425
[_get_item_kind]: https://github.com/microsoft/vscode-python/blob/411d1d4ae7e3884a8ef9294a0aa1688dc05d0f37/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py#L530-L540
@github-actions github-actions bot removed the needs PR label Aug 9, 2022
@karrtikr karrtikr added the needs PR Ready to be worked on label Aug 9, 2022
@eleanorjboyd
Copy link
Member

Hello! We have just finished our testing rewrite and are beginning the roll out to users. If you are able, it would be very helpful to know if your issue still exists on the rewrite! To use the rewrite yourself just add ”python.experiments.optInto": ["pythonTestAdapter"] to your user settings. You can confirm you have the rewrite enabled by setting your log level to trace, via the Developer: Set Log Level command in the command palette. Then check to see if Experiment 'pythonTestAdapter' is active is in your python logs.

Thanks!

@svenaron
Copy link

svenaron commented Nov 16, 2023 •
edited
Loading

Hello! We have just finished our testing rewrite and are beginning the roll out to users. If you are able, it would be very helpful to know if your issue still exists on the rewrite! To use the rewrite yourself just add ”python.experiments.optInto": ["pythonTestAdapter"] to your user settings. You can confirm you have the rewrite enabled by setting your log level to trace, via the Developer: Set Log Level command in the command palette. Then check to see if Experiment 'pythonTestAdapter' is active is in your python logs.

Thanks!

@eleanorjboyd
I can confirm that the issue is still present, even with pythonTestAdapter enabled.
The tests are shown in the output from the pytest collect call, but not in the test-panel.

@eleanorjboyd
Copy link
Member

Thank you for taking a look. I will investigate this shortly, thanks!

@eleanorjboyd eleanorjboyd self-assigned this Dec 4, 2023
@eleanorjboyd eleanorjboyd added the needs community feedback Awaiting community feedback label Dec 20, 2023
@github-actions GitHub Actions
Copy link

Thanks for the feature request! We are going to give the community 60 days from when this issue was created to provide 7 👍 upvotes on the opening comment to gauge general interest in this idea. If there's enough upvotes then we will consider this feature request in our future planning. If there's unfortunately not enough upvotes then we will close this issue.

@eleanorjboyd
Copy link
Member

Need community feedback here on if this is still a relevant ask people want for the extension, thanks!

@brettcannon
Copy link
Member

Thank you for submitting your feature request and everyone who considered it! Unfortunately, this issue did not receive enough votes over the allotted time, and so we are closing the issue.

@brettcannon brettcannon closed this as not planned Won't fix, can't repro, duplicate, stale Feb 22, 2024
@drothlis drothlis mentioned this issue Mar 12, 2024
Open
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@eleanorjboyd eleanorjboyd

Labels
area-testing feature-request Request for new features or functionality needs community feedback Awaiting community feedback needs PR Ready to be worked on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@brettcannon @zztalker @ericsnowcurrently @karrtikr @eleanorjboyd @svenaron @luabud