"Test result not found" if cwd is a subdirectory in the workspace

[adri0]

VS Code version

1.62.2

Extension version

v2021.11.1422169775

OS type

macOS

OS version

10.15.7

Python distribution

python.org

Python version

Python 3.9.5

Language server

None or Disabled

Expected behaviour

Test results should show after running tests using Testing sidebar.

Actual behaviour

Exception raised after test finished running:

Test result not found for: ./tests/test_mod.py::test_1

Steps to reproduce

Structure of the repository that reproduces the issue:

.
├── .gitignore
├── README.md
├── backend
│   ├── .python-version
│   ├── Pipfile
│   ├── Pipfile.lock
│   ├── src
│   │   └── testmod
│   │       └── __init__.py
│   └── tests
│       ├── conftest.py
│       └── test_mod.py
└── test-pytest.code-workspace

I included in the repository the settings.json used. That shows:

{
    "python.defaultInterpreterPath": "/Users/adri0/.local/share/virtualenvs/backend-kdNxFSnI",
    "python.terminal.activateEnvironment": true,
    "python.terminal.activateEnvInCurrentTerminal": true,
    "python.testing.cwd": "${workspaceFolder}/backend",
    "python.testing.pytestEnabled": true,
    "python.testing.unittestEnabled": false,
    "python.testing.pytestArgs": [
        "tests/"
    ]
}

I noticed that if I change the cwd to ${workspaceFolder} and move the tests folder to the repository root, it works.

Logs

cwd: ~/code/test-pytest-clone/backend
> ~/.local/share/virtualenvs/backend-kdNxFSnI/bin/python ~/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/code/test-pytest-clone/backend -s --cache-clear tests/

Code of Conduct

• I agree to follow this project's Code of Conduct

[karthiknadig]

@adri0 Here are the changes I made to get this working see if it works for you:
This is the structure:

The "tests" are out side of the src folder.

Remove the settings.json, as you can see I don't have a .vscode folder.

This is my workspace.code-workspace:

{
    "folders": [
        {
            "path": "."
        }
    ],
    "settings": {
        "python.defaultInterpreterPath": "/Users/adri0/.local/share/virtualenvs/backend-kdNxFSnI",
        "python.terminal.activateEnvironment": true,
        "python.terminal.activateEnvInCurrentTerminal": true,
        "python.testing.cwd": "${workspaceFolder}",
        "python.testing.pytestEnabled": true,
        "python.testing.unittestEnabled": false,
        "python.testing.pytestArgs": ["tests"]
    }
}

I did some minor tweaks to the cwd and pytestAtgs settings.

It seems to be working:

[karthiknadig]

Let me know if the above changes work for you.

[adri0]

The problem does go away when I change the folder structure for the one proposed. The issue with it is that in some real life projects it's not an option for me to have the tests folder directly under the root directory, as the Python code is a component of a bigger project.

Thus I thought, why not change the python.testing.cwd config to match the folder structure, given I'm allowed to change it. When I notice this prevents the tests to work properly I thought this behaviour could be a bug.

[adri0]

Thanks about the settings in the workspace file tip. It does simplify things. I'll update the sample project.

[adri0]

I made an edit in the first comment. The actual folder structure should be ./backend/tests instead of ./tests.

[stilren]

I am also seeing issues with this. We have a very nested directory structure since we are deploying aws lambdas and each lambda has its own test folder

├── .gitignore
├── README.md
├── app
│   ├── lambda1
│   │   └── src
│   │       └── handler1.py
│   └── tests
│       ├── conftest.py
│       └── test_handler1.py
│   ├── lambda2
│   │   └── src
│   │       └── handler2.py
│   └── tests
│       ├── conftest.py
│       └── test_handler2.py

setting
"python.testing.cwd": "${workspaceFolder}/app"

throws the same type of error.

Here is a minimal example showing this error:

Folder structure

├── src
│   └── test_example.py

contents of test_example.py

from unittest import TestCase

class ExampleTest(TestCase):
    def test_example_test(self):
        assert 1 == 1

contents of settings.json

{
    "python.testing.unittestArgs": [
        "-v",
        "-s",
        ".",
        "-p",
        "test_*.py"
    ],
    "python.testing.pytestEnabled": true,
    "python.testing.unittestEnabled": false,
    "python.testing.cwd": "${workspaceFolder}/src"
}

[tinix0]

I can also confirm the same. We have tests in a subdirectory of the workspace with code directly in root.
Settings:
"python.testing.cwd": "${workspaceFolder}/unit_tests"

[changeling]

I'm having this same (or similar) issue, though my tests are not in a subdirectory.

Clicking on Run Tests in the test panel yields Test result not found for: ./tests/[test_name].py::test_no_argument.

Here's the twist, though: Clicking on Rerun Failed Tests after the fail yields a complete, successful run.

Subsequent to that, clicking on Run Tests fails again. So I can successfully run my tests, but only by clicking on Rerun Failed Tests after first attempting to run.

In addition, running the individual tests by clicking on their respective Run Test buttons works fine.

My directory looks like this:
│── .gitignore
│── README.md
│── src
│ └── example.py
│── tests
│ └── test_example.py

---

VS Code version
Version: 1.64.2
Python extension version
v2022.0.1814523869
OS: Darwin x64 19.6.0 (macOS 10.15.7)

[eleanorjboyd]

Hello! We have just finished our testing rewrite and are beginning the roll out to users. If you are able, it would be very helpful to know if your issue still exists on the rewrite! To try it yourself, add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

You can confirm you have the rewrite enabled by setting "python.analysis.logLevel": "Trace", in your user settings then check for Experiment 'pythonTestAdapter' is active in your python logs.

Let me know if the rewrite fixes your issue. Thanks!

[eleanorjboyd]

closing as there has been no activity on this issue in a month, please comment to reopen. Thanks

[Rafq77]

Hi, it seems to work just fine, however I have some problems with tests that use MagicMock as file descriptor instance.

I could test the fixes by enabling in my user settings.json the "python.experiments.optInto": ["pythonTestAdapter"] successfully. However, "python.analysis.logLevel" is an "Unknown Configuration Setting" for my vscode instance.

My test setup is following:

VS Code version

1.84.0

Extension version

v2023.21.13101009

OS type & version

Windows 10

Python version

Python 3.11.2

Given following test:

from unittest.mock import MagicMock

def test_test_magic_mock_instance():
    tmp = MagicMock()
    openHexOutputFile = open(tmp , "a", encoding="utf-8");
    openHexOutputFile.close()

The process crashes internally.

Note: ⚠ That, if this test is run in debug mode 🐞 with/-out breakpoints, it completes flawlessly ✅.

Due to the not working analysis.logLevel setting on my side, I don't have elaborate logs, except for those pluggy(?) errors:

CLIENT: Server listening on port 61249...
Received JSON data in run script
============================= test session starts =============================
platform win32 -- Python 3.11.2, pytest-7.4.2, pluggy-1.3.0
rootdir: c:\svn\...
plugins: cov-4.1.0, mock-3.11.1
collected 1 item

\tools\py.build\_tests\test_section.py Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\main.py", line 271, in wrap_session
    session.exitstatus = doit(config, session) or 0
                         ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\main.py", line 325, in _main
    config.hook.pytest_runtestloop(session=session)
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 152, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\main.py", line 350, in pytest_runtestloop
    item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 152, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\runner.py", line 114, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\runner.py", line 133, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\runner.py", line 226, in call_and_report
    hook.pytest_runtest_logreport(report=report)
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\terminal.py", line 599, in pytest_runtest_logreport
    self._tw.write(letter, **markup)
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\_io\terminalwriter.py", line 155, in write
    self._file.write(msg)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\main.py", line 291, in wrap_session
    config.notify_exception(excinfo, config.option)
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\config\__init__.py", line 1108, in notify_exception
    res = self.hook.pytest_internalerror(excrepr=excrepr, excinfo=excinfo)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\terminal.py", line 527, in pytest_internalerror
    self.write_line("INTERNALERROR> " + line)
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\terminal.py", line 482, in write_line
    self.ensure_newline()
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\terminal.py", line 448, in ensure_newline
    self._tw.line()
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\_io\terminalwriter.py", line 171, in line
    self.write("\n")
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\_io\terminalwriter.py", line 155, in write
    self._file.write(msg)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\...\.vscode\extensions\ms-python.python-2023.21.13101009\pythonFiles\vscode_pytest\run_pytest_script.py", line 67, in <module>
    pytest.main(arg_array)
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\config\__init__.py", line 169, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\main.py", line 318, in pytest_cmdline_main
    return wrap_session(config, _main)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\main.py", line 306, in wrap_session
    config.hook.pytest_sessionfinish(
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pluggy\_callers.py", line 130, in _multicall
    teardown[0].send(outcome)
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\terminal.py", line 858, in pytest_sessionfinish
    self._tw.line("")
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\_io\terminalwriter.py", line 171, in line
    self.write("\n")
  File "C:\Program Files\Python311\Lib\site-packages\_pytest\_io\terminalwriter.py", line 155, in write
    self._file.write(msg)
OSError: [Errno 9] Bad file descriptor
Finished running tests!

[ajoga]

it would be very helpful to know if your issue still exists on the rewrite! To try it yourself, add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

Hi @eleanorjboyd -- this, followed by a window reload just fixed the issue that seems more or less the same as the author or this ticket. Thank you!

[eleanorjboyd]

Hello @Rafq77, sorry for the delay as I was on vacation.

Could you open up a separate issue for the bug you are seeing? It is very interesting but seems less related to the current thread and a new thread would allow me to analyze it further. Thanks!