Escape pytest in zsh terminal when debuging the tests

[cranechu]

Environment data

• VS Code version: 1.32.3
• Extension version (available under the Extensions sidebar): 2019.2.5558
• OS and version: Fedora 29
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.7.2 64-bit
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: N/A

Expected behaviour

Debug parameterized pytest by click the "bug" logo in the test tree view, the debugger should start.

Actual behaviour

Debug parameterized pytest (e.g. test_qpair_different_size[5] below), but cannot find the test.
For normal pytest cases, the debug just works, e.g. test_create_device_invalid below.

Steps to reproduce:

1. launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "debug pynvme script",
            "type": "python",
            "request": "test",
            "console": "integratedTerminal", 
            "sudo": true
        }
    ]
}

Logs

~/pynvme▶ cd /home/cranechu/pynvme ; env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 sudo /usr/bin/python3 /home/cranechu/.vscode/extensions/ms-python.python-2019.2.5558/pythonFiles/ptvsd_launcher.py --default --client --host localhost --port 40837 /home/cranechu/.vscode/extensions/ms-python.python-2019.2.5558/pythonFiles/testlauncher.py /home/cranechu/pynvme pytest --junitxml=/tmp/tmp-3191CZSiAJnJWSOU.xml --pciaddr=01:00.0 driver_test.py::test_qpair_different_size[5] 
zsh: no matches found: driver_test.py::test_qpair_different_size[5]

~/pynvme▶ cd /home/cranechu/pynvme ; env PYTHONIOENCODING=UTF-8 PYTHONUNBUFFERED=1 sudo /usr/bin/python3 /home/cranechu/.vscode/extensions/ms-python.python-2019.2.5558/pythonFiles/ptvsd_launcher.py --default --client --host localhost --port 38197 /home/cranechu/.vscode/extensions/ms-python.python-2019.2.5558/pythonFiles/testlauncher.py /home/cranechu/pynvme pytest --junitxml=/tmp/tmp-3191X7ih17lf4bBg.xml --pciaddr=01:00.0 driver_test.py::test_create_device_invalid 
pydev debugger: warning: trying to add breakpoint to file that does not exist: /home/cranechu/pynvme/scripts/test_nvme_spec.py (will have no effect)
================================== test session starts ==================================
platform linux -- Python 3.7.2, pytest-4.1.1, py-1.7.0, pluggy-0.8.1 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/cranechu/pynvme, inifile: pytest.ini
plugins: cov-2.6.1
collected 1 item    
...

[DonJayamanne]

Please could you test this with the latest version of the extension and confirm it exists.
If it does then please provide the following info:

• A sample repo (zip file, github, etc) to repro the issue
• Steps to reproduce the issue (your current steps do not contain sufficient information to repro the issue, just contains a file).

Debug parameterized pytest (e.g. test_qpair_different_size[5] below), but cannot find the test.
For normal pytest cases, the debug just works, e.g. test_create_device_invalid below.

Please can you be more specific with the Actual Behavior. What is happening today.
Please could you provide screenshots of the errors.

[cranechu]

Hi,

I repeated the test with python extension version 2019.3.6352, and the symptoms are the same.
Here is a simplified repro for you to reproduce. pytest_debug_demo.tar.gz

Steps:

1. code . to open the vscode, and then open the python file: test_scripts.py. This file contains 2 test functions, one is parameterized multiple cases, another is the normal single test.
2. debug the test case test_ioworker_iops_single(), works fine.
3. debug the test case test_ioworker_iops_multiple_queue, and choose test_ioworker_iops_multiple_queue[1] in the opened dialog, then debug fail shown in this screenshot:
   
4. run tests work for both test functions.
   
5. another thing, may be related, that actually I cannot jump to test source code by clicking the open icon of the parameterized test cases. While it does work on that single test case.
   

In recent releases, I found python extension becoming much much better than before on pytest functions! Thanks very much for your effort! It helps me make the choice between vscode and pycharm. :)

Thanks!

[DonJayamanne]

Debugging parameterized tests works for me, see below
Please could you install the latest version of the extension and try this out.

another thing, may be related, that actually I cannot jump to test source code by clicking the ope

This issue is being tracked here #4469

[cranechu]

Hi,

I noticed that you are using mac os and the shell should be bash. I am using zsh in Fedora.

I added the configuration below, and it works!

"terminal.integrated.shell.linux": "/bin/bash"

But I still prefer to use zsh, so I did some experiments like this:

It seems quotation mark is required on "file_name::function_name[parameter]" in zsh by its default configuration.

Thanks!

[evilmav]

Still experiencing this issue as of v2020.8.103604. A usable workaround is adding unsetopt nomatch to you ~/.zshrc.

[mchccc]

Still experiencing this too.. any chance for an update after @cranechu's comment?

[akaihola]

@evilmav is there any way to make unsetopt nomatch only effective inside VSCode, but not in my standard terminal?

[jaraqueffdc]

Hi, I am also having this issue where I always get a message saying that the test could not be found. Running outside debugging it works just great but not being able to debug a test takes away very much needed functionality.

When they are launched and fail VSCode reports a timeout issue as well and I can never run them again unless I restart vscode which makes the entire process very cumbersome.

Do you have any update in this?

I am u sing zsh for the terminal and my python extension version is v2021.4.765268190.

Here is test case that has the same properties as the tests I cannot run. Probably something simpler would reproduce the issue as well but I wanted to be as close as possible to what I have

def val1():
    yield "val1"


def val2():
    yield "val1 and val2"


@pytest.fixture(params=["val1", "val2"])
def fixture_factory(request):
    values = {
        "val1": val1,
        "val2": val2,
    }
    yield from values[request.param]()


def test_factory(fixture_factory):
    print("Entering test")
    assert isinstance(fixture_factory, str)

[eleanorjboyd]

Hello! We have finished our testing rewrite and it is enabled for all users.You can confirm you have the rewrite enabled by setting your log level to trace in your command palette then check for Experiment 'pythonTestAdapter' is active in your python logs.

Let me know if the rewrite fixes your issue. Thanks!

[github-actions]

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.

Happy Coding!