Python.testing.pytestArgs does not work with option -m

[mondwan]

Type: Bug

Behaviour

Expected vs. Actual

For settings.json

{
"python.testing.pytestArgs": ["-m", "lazy"]
}

Expect to see vscode to run pytest with argument in TEST RESULT UI

Running test with arguments: --rootdir /home/vdc/git/six --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-1147287WvATg175YF9w.xml -m lazy

But, -m is gone

Running test with arguments: --rootdir /home/vdc/git/six --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-1147287NQVjhdMkoSyI.xml

However, if you do something like that, it will work

{
"python.testing.pytestArgs": ["-m lazy"]
}

Steps to reproduce:

1. Set -m in the settings.json

{
"python.testing.pytestArgs": ["-m", "lazy"]
}

2. Run the vscode test

Personally, I clone a open source project six to replicate this issue. HOpefully, this can be replicated with any python project

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.8.10
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Jedi

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-08-10 13:28:40.743 [info] Discover tests for workspace name: six - uri: /home/vdc/git/six
2023-08-10 13:28:40.765 [info] > ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2023-08-10 13:28:40.765 [info] cwd: .
2023-08-10 13:28:45.858 [info] Discover tests for workspace name: six - uri: /home/vdc/git/six
2023-08-10 13:28:45.858 [warning] Unknown command line option passed into args parser for tests '-m lazy'. Please report on https://github.com/Microsoft/vscode-python/issues/new
2023-08-10 13:28:45.858 [warning] Unknown command line option passed into args parser for tests '-m lazy'. Please report on https://github.com/Microsoft/vscode-python/issues/new
2023-08-10 13:28:45.884 [info] > ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear "-m lazy"
2023-08-10 13:28:45.884 [info] cwd: .

User Settings

languageServer: "Jedi"

linting
• pycodestyleEnabled: true
• pydocstyleEnabled: true

formatting
• provider: "black"
• blackArgs: "<placeholder>"

testing
• promptToConfigure: false
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• autoTestDiscoverOnSaveEnabled: false

Extension version: 2023.14.0
VS Code version: Code 1.81.0 (6445d93c81ebe42c4cbd7a60712e0b17d9463e97, 2023-08-02T12:37:13.485Z)
OS version: Windows_NT x64 10.0.19044
Modes:
Remote OS version: Linux x64 5.4.0-148-generic
Remote OS version: Linux x64 5.4.0-148-generic
Remote OS version: Linux x64 5.4.0-148-generic

[eleanorjboyd]

Hello! Could you try this on the rewrite, we have a new testing architecture. To try it yourself, you need to add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

You can confirm you have the rewrite enabled by setting "python.analysis.logLevel": "Trace", in your user settings then check for Experiment 'pythonTestAdapter' is active in your python logs.

Let me know if the rewrite fixes your issue. Thanks!

[mondwan]

Hi, I tried with following

{
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": ["-m", "lazy"],
    "python.experiments.optInto": ["pythonTestAdapter"],
    "python.analysis.logLevel": "Trace"
}

But, it sounds like the experiment option cannot be active, and the option does not work as expected neither

2023-08-21 11:22:11.849 [info] Telemetry is disabled
2023-08-21 11:22:11.849 [info] Experiments are disabled, only manually opted experiments are active.
2023-08-21 11:22:11.853 [info] Test server listening.
2023-08-21 11:22:11.959 [info] > ./venv/bin/python -I ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/interpreterInfo.py
2023-08-21 11:22:12.034 [info] Python interpreter path: ./venv/bin/python
2023-08-21 11:22:12.552 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.934 [info] Starting Jedi language server for six.
2023-08-21 11:22:28.337 [info] Discover tests for workspace name: six - uri: XXXXXXXXXXXXXXXXXXXXXXXX
2023-08-21 11:22:28.412 [info] > . ./venv/bin/activate && echo 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' && python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/printEnvVariables.py
2023-08-21 11:22:28.412 [info] shell: bash
2023-08-21 11:22:28.476 [info] > ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2023-08-21 11:22:28.476 [info] cwd: .

[eleanorjboyd]

hm interesting. Did you reload your vscode windows? Also, you set it in your user settings json not your workspace one correct?

[mondwan]

As I am using a remote vscode, I have tried to edit my host's settings.json.

..
    "python.experiments.optInto": ["pythonTestAdapter"],
..

Then, I tried to reload the vscode. But, the results are the same, which is does not work as expected.

[mondwan]

Here is how I reproduce this bug in a public repository

Clone a public repository https://github.com/benjaminp/six
Setup the settings.json as described
Run the tests as described

Hope this helps

[eleanorjboyd]

related to #21845. We will need to overhaul args now that we have more flexibility with our architecture. This issue will be resolved by that. Thanks

[eleanorjboyd]

Hi! Have you had any luck with this issue recently? We just expanded the experiment to include everyone so you should be on the rewrite (without the need for the setting). From your logs they always included 2023-08-21 11:22:28.476 [info] > ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear which is the path to the old version of the code. Let me know if there are any changes, the overhaul to the args will also fix the problem but in the meantime it might help. Thanks

[mondwan]

Is it released? I just tried. It still does not work

I am using ms-python.python 2023.20.0

[eleanorjboyd]

Are you still seeing ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py in your ags? It should be enabled now unless you have a specific setting disabling all experiments.

[mondwan]

Not sure I am following. Can you list the full settings required for testing your fix? 2ndly, would you mind to list the expectations for the fix? As I tried the settings listed in this thread, option m still does not work if it is not specify as one single string

…

On Thu, 7 Dec 2023, 17:59 Eleanor Boyd, ***@***.***> wrote: Are you still seeing ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py in your ags? It should be enabled now unless you have a specific setting disabling all experiments. — Reply to this email directly, view it on GitHub <#21798 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5YBI3BV5S4IMQZA6BK2QTYIH7Y5AVCNFSM6AAAAAA3LNVSNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBVHA2TCNRQHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[github-actions]

Hey @eleanorjboyd, this issue might need further attention.

@mondwan, you can help us out by closing this issue if the problem no longer exists, or adding more information.

[robertdj]

Any news on this issue?

I have noticed that I cannot debug tests if I use ["-m lazy"], but I can if I use ["-m", "lazy"].

[eleanorjboyd]

All args must be in separate strings in the array so ["-m", "lazy"] is correct, if that is working then this is the desired behavior. @mondwan, are you seeing this too? If so this issue is resolved and I will close, thanks

[robertdj]

Oops! That was lacking some context.

If I use ["-m", "lazy"] (or in my case ["-m", "not lazy"]) then the mark filtering is NOT respected, but I CAN debug.

If I use ["-m lazy"] then the mark filtering IS respected, but I CANNOT debug.

[mondwan]

hi @robertdj
Then, there are 2 bugs. mine one is the 1st situation. In other words, the mark filtering does not work as expected.
As suggested by @eleanorjboyd , ["-m", "lazy"] should be the expected usage. But, mark filtering in this context does not work as expected

hi @eleanorjboyd
I have double checked just now. It still does not work as expected. Can you show your settings & results?

[eleanorjboyd]

apologies I got mixed up, this is waiting on #21845 and that proposal will fix the incorrect arg parsing. Sorry and thank you for your patience!

[eleanorjboyd]

We removed arg parsing for pytest so this should no longer be an issue. I gave it a try and it seems to be resolved, please comment if you are still having an issue and I can re-open. Thanks!