Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest parameterized tests with randomly generate names #22648

Closed
cquick01 opened this issue Dec 13, 2023 · 4 comments
Closed

pytest parameterized tests with randomly generate names #22648

cquick01 opened this issue Dec 13, 2023 · 4 comments
Assignees
Labels
area-testing info-needed Issue requires more information from poster triage-needed Needs assignment to the proper sub-team

Comments

@cquick01
Copy link

cquick01 commented Dec 13, 2023

Type: Bug

Behaviour

Expected vs. Actual

When running pytest via VSCode, I get the following errors

Running tests (pytest): /home/user/test/tests/test_bytes.py::test_randbytes
Running test with arguments: --rootdir /home/user/test --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-522739qc0UGxP1z6FD.xml --capture no ./tests/test_bytes.py::test_randbytes
Current working directory: /home/user/test
Workspace directory: /home/user/test
Run completed, parsing output
Test result not found for: ./tests/test_bytes.py::test_randbytes[b8\xef\xef]

Test result not found for: ./tests/test_bytes.py::test_randbytes[\xb4d\xed\xaek\xac\xdbR\xde\xe2\x87\xcc5\x9c]\x95\x02.\xac\xd1)\x8a\xb3\xaf\x15\x87\xad\x83c\x0b\x94Z\x8d\x03\xcd<\xd6\xa0\xe9\xba\x11\x95]

Test result not found for: ./tests/test_bytes.py::test_randbytes[5?5\x9cIGQG\xba\x8a\xdb\xf7\xc8\xec k\x95\x08\xbb \x8c\x1eBb\xd33\x93\x1c\x93\xe6\xc2\x08\x7f\xd1\xa7\x95\xf5m\xca\xaa\xe2\x90\x8c\x97\x80\xfe\xd0\xd2#\xbdI\x1dV3>\x8e7\x01\xffg\r]

Test result not found for: ./tests/test_bytes.py::test_randbytes[k[\x02M\xe2\r*\xcd4\xc2Q+\xadbqP\xca\xac\xc4]

Test result not found for: ./tests/test_bytes.py::test_randbytes[\xdb\x1d\xe8\x04\n\x9b\x0b/.\xd55\xd2\x92\xbf\x05/\xc1\xa0\x06\x95\xd2s\x18\xd1\xed'\xd9Cu\x02S\xe1\x0b\xc4\xa6\x00)\xda\xf1]

Test result not found for: ./tests/test_bytes.py::test_randbytes[\xe4\x94`\xd0\xb9V\x97g%\xafa\xda\xc8\xd1\xbd\xd7\xc5Z\xf7\xb7\xdf\xbdd]

Test result not found for: ./tests/test_bytes.py::test_randbytes[\xb2\x8f]

Test result not found for: ./tests/test_bytes.py::test_randbytes[z:\x1f\xccq`\xd6\x88\xef\x8a\xbc*\x8f\x95\x8e\x8c\xf9O\xf1\xdbo(]

Test result not found for: ./tests/test_bytes.py::test_randbytes[\xd5h[\x7f\xe2\xf6m\xf1]

Test result not found for: ./tests/test_bytes.py::test_randbytes[_\x82p\xac\x07\xc1\xdbO\n\xe1<\x19\xb2\xbbW\x18\x95\x02H\xc8]

Total number of tests expected to run: 10
Total number of tests run: 0
Total number of tests passed: 0
Total number of tests failed: 0
Total number of tests failed with errors: 0
Total number of tests skipped: 0
Total number of tests with no result data: 10
Finished running tests!

But the debug window shows that it passes

============================= test session starts ==============================
platform linux -- Python 3.11.3, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/user/test
collected 10 items

tests/test_bytes.py ..........

------------- generated xml file: /tmp/tmp-522739qc0UGxP1z6FD.xml --------------
============================== 10 passed in 0.10s ==============================

And it works fine when running on command line.

Steps to reproduce:

  1. Setup Environment
.
├── poetry.lock
├── pyproject.toml
└── tests
    └── test_bytes.py

test_bytes.py

#!/usr/bin/env python3
import random

import pytest

RANDOM_BYTES = [random.randbytes(random.randint(0, 64)) for _ in range(10)]


@pytest.mark.parametrize("data", RANDOM_BYTES)
def test_randbytes(data):
    foo = data
    assert foo == data

pyproject.toml

[tool.poetry]
name = "test"
version = "0.1.0"
description = ""
authors = ["User <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"
pytest = "^7.4.3"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
  1. Install with poetry
pip install poetry  # if needed
poetry lock
poetry install
  1. Configure Python interpreter in VSCode
# to get path of environment
poetry env info -p
  1. Run tests, errors occur. Running pytest in terminal passes.

Edit: Also, when adding this test with ints, most fail the same way, but two of them seem to pass.

image

But the debug window still shows all tests pass, 20 this time

============================= test session starts ==============================
platform linux -- Python 3.11.3, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/cquick/test
collected 20 items

tests/test_bytes.py ....................

------------- generated xml file: /tmp/tmp-522739BQWWvKOgqoHN.xml --------------
============================== 20 passed in 0.10s ==============================

Diagnostic data

  • Python version (& distribution if applicable, e.g. Anaconda): 3.11.3
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Poetry
  • Value of the python.languageServer setting: Pylance
Output for Python in the Output panel (ViewOutput, change the drop-down the upper-right of the Output panel to Python)

2023-12-13 17:19:53.199 [debug] Clearing context for python dependencies not installed: /home/user/test/pyproject.toml
2023-12-13 17:19:53.202 [debug] Found cached env for /home/user/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3
2023-12-13 17:19:53.202 [debug] Found cached env for /home/user/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3
2023-12-13 17:19:53.219 [debug] Found cached env for /home/user/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3
2023-12-13 17:19:53.234 [info] > ~/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3 -m pytest --rootdir . --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-5227399oU3WLyyfScP.xml
2023-12-13 17:19:53.234 [info] cwd: .
2023-12-13 17:19:57.440 [debug] Clearing context for python dependencies not installed: ms-python.python.Python
2023-12-13 17:19:57.440 [debug] Found cached env for /home/user/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3
2023-12-13 17:20:03.697 [debug] Clearing context for python dependencies not installed: /home/user/test/pyproject.toml
2023-12-13 17:20:03.700 [debug] Found cached env for /home/user/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3
2023-12-13 17:20:11.268 [debug] Clearing context for python dependencies not installed: ms-python.python.Python
2023-12-13 17:20:11.268 [debug] Found cached env for /home/user/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3

User Settings


languageServer: "Pylance"

testing
• pytestEnabled: true

Extension version: 2023.23.13471011
VS Code version: Code - Insiders 1.86.0-insider (f87fb292c3af22e51a7f5c628496cd8d6fd9ecd5, 2023-12-13T08:07:18.340Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Remote OS version: Linux x64 5.15.133.1-microsoft-standard-WSL2

System Info
Item Value
CPUs Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 x 2592)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
Load (avg) undefined
Memory (System) 31.70GB (5.56GB free)
Process Argv
Screen Reader no
VM 0%
Item Value
Remote WSL: Ubuntu-22.04
OS Linux x64 5.15.133.1-microsoft-standard-WSL2
CPUs Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 x 2592)
Memory (System) 11.68GB (6.36GB free)
VM 0%
@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Dec 13, 2023
@eleanorjboyd
Copy link
Member

Hello, it looks like you are not on the new testing rewrite. Could you enable that and try this again?

It should be turned on for everyone but you might have disabled all experiments so could you check there? If not you could try adding this to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"]. You can check if its enabled by looking at Experiment 'pythonTestAdapter' is active in your python logs. If you are unable to see the log for the experiment change your log level to info or trace and it should show up (can be done via the command palette). Thanks!

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Dec 14, 2023
@cquick01
Copy link
Author

Thanks for the response @eleanorjboyd . I enabled the pythonTestAdapter experiment but am still running into issues running the tests.

I see that the experiment is now activated

2023-12-15 09:48:03.524 [info] Telemetry level is off
2023-12-15 09:48:03.524 [info] Experiments are disabled, only manually opted experiments are active.
2023-12-15 09:48:03.524 [info] Experiment 'pythonTestAdapter' is active
2023-12-15 09:48:03.722 [info] Default formatter is set to null for workspace /home/user/test
2023-12-15 09:48:04.805 [info] Test server listening.
2023-12-15 09:48:05.027 [info] > ~/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3 -I ~/.vscode-server-insiders/extensions/ms-python.python-2023.23.13481009/pythonFiles/get_output_via_markers.py ~/.vscode-server-insiders/extensions/ms-python.python-2023.23.13481009/pythonFiles/interpreterInfo.py
2023-12-15 09:48:05.667 [info] Python interpreter path: ~/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/python3
2023-12-15 09:48:05.968 [info] > conda info --json
2023-12-15 09:48:06.339 [info] > . ~/.cache/pypoetry/virtualenvs/test-zxyBamcN-py3.11/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode-server-insiders/extensions/ms-python.python-2023.23.13481009/pythonFiles/printEnvVariables.py
2023-12-15 09:48:06.339 [info] shell: bash
2023-12-15 09:48:07.403 [info] Starting Pylance language server.

The tests still show errors, although they seem to be different now

CLIENT: Server listening on port 41559...
Received JSON data in run script
Running pytest with args: ['-p', 'vscode_pytest', '--rootdir', '/home/user/test', '/home/user/test/tests/test_bytes.py::test_randints[17]', '/home/user/test/tests/test_bytes.py::test_randints[46]', '/home/user/test/tests/test_bytes.py::test_randints[6]', '/home/user/test/tests/test_bytes.py::test_randints[18]', '/home/user/test/tests/test_bytes.py::test_randints[0]', '/home/user/test/tests/test_bytes.py::test_randints[48]', '/home/user/test/tests/test_bytes.py::test_randints[14]', '/home/user/test/tests/test_bytes.py::test_randints[60]', '/home/user/test/tests/test_bytes.py::test_randints[39]', '/home/user/test/tests/test_bytes.py::test_randints[64]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\x80\\xde}=\\x8c2/7\\x00Y\\x10`\\x0b\\xce%\\xfd\\xbd\\xd0_\\xee\\x11\\xe8rW\\x8c\\xb9\\xf0N\\xa11<f\\xc9u\\xb3\\x92\\x18\\x0c\\xeb\\x99\\x136?\\x13\\xc7\\xda\\xbe\\xee\\xc1t\\x87\\xeaE\\xa7\\xebk\\x14\\x86\\xcdv]', '/home/user/test/tests/test_bytes.py::test_randbytes[c\\xd9\\xd9\\x98|e`V\\xc3\\xce\\x04N\\xfc*\\xfc)o\\xb5\\xa6\\xe6\\xedJ\\x0eo\\x94\\xea\\x17\\xef]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\x9b\\xe4\\xf63yd\\xe1)\\x02\\x87\\x17\\xd3\\xc4\\xc7M\\xacc\\xab\\x1cdJ\\xca\\x04\\\\x81\\n\\xca.\\xe7\\x9f\\x13\\xfd\\xe3\\x86\\xa2c\\xe2\\xfe\\xaf]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\x05\\xd5\\xed\\x12\\xa4!e\\xaf\\x16$\\xfb"<\\x91\\xce\\xd9\\xb9\\xdb`\\xbe\\xfd:\\xf0\\xd8/,\\x84\\xbdC\\xf7\\xdd\\x0c\\x94?\\xe4\\xe0\\xbe]\\xa2\\xb53\\xf6\\xb80\\x93\\xfd\\xbabC\\xfc\\xba\\xe5s<\\x00d]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\xb8\\x0e\\xfcs\\xa2]\\xfe\\xb2\\xbe\\x06\\xd4\\x83\\x8e\\x9a\\xc1\\xee\\xd7K\\nL=^\\xb6\\x99\\xce1N\\xf5\\x06\\x9f\\xebM\\xf0\\xfb\\x9c\\xc0\\x81\\x04i\\x10\\x7f\\xa9]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\x92k\\x11\\x1c\\xb0m\\x89\\x16j\\x02"\\xf3\\xf2\\xc9\\xcb\\x9aG]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\xec"\\nZ\\xbc\\xd5T\\xb1\\xeczX\\xd1\\xee]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\xe2\\xa4\\x84&\\xb2[Hk\\xc67\\xaf\\x13\\x96\\x18\\xa6\\xbd\\x0bC\\x1cS\\xd1\\xde\\x9e\\xfeJ\\x87,\\x00L\\xefr\\xc3\\xfb-\\x81\\xdb\\xf9\\xc5]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\xdf\\xab)u\\x14\\xe2\\x8d\\xe0\\xd8\\x13\\xb5\\xddn\\xe9\\xe3K\\xf0E\\xb1]', '/home/user/test/tests/test_bytes.py::test_randbytes[\\x00i\\xb2\\x08\\x86\\x99\\x98"\\x1bw<\\xf4\\x84[\\xc4\\xb9\\x81\\x9f\\x00q\\xaes\\xce\\xfd\\xb4^\\xbe\\x12\\x8e\\xdcv\\xff|\\x06\\x9d\\x97]']
============================= test session starts ==============================
platform linux -- Python 3.11.3, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/user/test
collected 2 items

============================ no tests ran in 0.01s =============================
ERROR: not found: /home/user/test/tests/test_bytes.py::test_randints[17]
(no name '/home/user/test/tests/test_bytes.py::test_randints[17]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randints[46]
(no name '/home/user/test/tests/test_bytes.py::test_randints[46]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randints[6]
(no name '/home/user/test/tests/test_bytes.py::test_randints[6]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randints[0]
(no name '/home/user/test/tests/test_bytes.py::test_randints[0]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randints[48]
(no name '/home/user/test/tests/test_bytes.py::test_randints[48]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randints[14]
(no name '/home/user/test/tests/test_bytes.py::test_randints[14]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randints[60]
(no name '/home/user/test/tests/test_bytes.py::test_randints[60]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randints[64]
(no name '/home/user/test/tests/test_bytes.py::test_randints[64]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\x80\xde}=\x8c2/7\x00Y\x10`\x0b\xce%\xfd\xbd\xd0_\xee\x11\xe8rW\x8c\xb9\xf0N\xa11<f\xc9u\xb3\x92\x18\x0c\xeb\x99\x136?\x13\xc7\xda\xbe\xee\xc1t\x87\xeaE\xa7\xebk\x14\x86\xcdv]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\x80\\xde}=\\x8c2/7\\x00Y\\x10`\\x0b\\xce%\\xfd\\xbd\\xd0_\\xee\\x11\\xe8rW\\x8c\\xb9\\xf0N\\xa11<f\\xc9u\\xb3\\x92\\x18\\x0c\\xeb\\x99\\x136?\\x13\\xc7\\xda\\xbe\\xee\\xc1t\\x87\\xeaE\\xa7\\xebk\\x14\\x86\\xcdv]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[c\xd9\xd9\x98|e`V\xc3\xce\x04N\xfc*\xfc)o\xb5\xa6\xe6\xedJ\x0eo\x94\xea\x17\xef]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[c\\xd9\\xd9\\x98|e`V\\xc3\\xce\\x04N\\xfc*\\xfc)o\\xb5\\xa6\\xe6\\xedJ\\x0eo\\x94\\xea\\x17\\xef]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\x9b\xe4\xf63yd\xe1)\x02\x87\x17\xd3\xc4\xc7M\xacc\xab\x1cdJ\xca\x04\\x81\n\xca.\xe7\x9f\x13\xfd\xe3\x86\xa2c\xe2\xfe\xaf]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\x9b\\xe4\\xf63yd\\xe1)\\x02\\x87\\x17\\xd3\\xc4\\xc7M\\xacc\\xab\\x1cdJ\\xca\\x04\\\\x81\\n\\xca.\\xe7\\x9f\\x13\\xfd\\xe3\\x86\\xa2c\\xe2\\xfe\\xaf]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\x05\xd5\xed\x12\xa4!e\xaf\x16$\xfb"<\x91\xce\xd9\xb9\xdb`\xbe\xfd:\xf0\xd8/,\x84\xbdC\xf7\xdd\x0c\x94?\xe4\xe0\xbe]\xa2\xb53\xf6\xb80\x93\xfd\xbabC\xfc\xba\xe5s<\x00d]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\x05\\xd5\\xed\\x12\\xa4!e\\xaf\\x16$\\xfb"<\\x91\\xce\\xd9\\xb9\\xdb`\\xbe\\xfd:\\xf0\\xd8/,\\x84\\xbdC\\xf7\\xdd\\x0c\\x94?\\xe4\\xe0\\xbe]\\xa2\\xb53\\xf6\\xb80\\x93\\xfd\\xbabC\\xfc\\xba\\xe5s<\\x00d]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\xb8\x0e\xfcs\xa2]\xfe\xb2\xbe\x06\xd4\x83\x8e\x9a\xc1\xee\xd7K\nL=^\xb6\x99\xce1N\xf5\x06\x9f\xebM\xf0\xfb\x9c\xc0\x81\x04i\x10\x7f\xa9]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\xb8\\x0e\\xfcs\\xa2]\\xfe\\xb2\\xbe\\x06\\xd4\\x83\\x8e\\x9a\\xc1\\xee\\xd7K\\nL=^\\xb6\\x99\\xce1N\\xf5\\x06\\x9f\\xebM\\xf0\\xfb\\x9c\\xc0\\x81\\x04i\\x10\\x7f\\xa9]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\x92k\x11\x1c\xb0m\x89\x16j\x02"\xf3\xf2\xc9\xcb\x9aG]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\x92k\\x11\\x1c\\xb0m\\x89\\x16j\\x02"\\xf3\\xf2\\xc9\\xcb\\x9aG]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\xec"\nZ\xbc\xd5T\xb1\xeczX\xd1\xee]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\xec"\\nZ\\xbc\\xd5T\\xb1\\xeczX\\xd1\\xee]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\xe2\xa4\x84&\xb2[Hk\xc67\xaf\x13\x96\x18\xa6\xbd\x0bC\x1cS\xd1\xde\x9e\xfeJ\x87,\x00L\xefr\xc3\xfb-\x81\xdb\xf9\xc5]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\xe2\\xa4\\x84&\\xb2[Hk\\xc67\\xaf\\x13\\x96\\x18\\xa6\\xbd\\x0bC\\x1cS\\xd1\\xde\\x9e\\xfeJ\\x87,\\x00L\\xefr\\xc3\\xfb-\\x81\\xdb\\xf9\\xc5]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\xdf\xab)u\x14\xe2\x8d\xe0\xd8\x13\xb5\xddn\xe9\xe3K\xf0E\xb1]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\xdf\\xab)u\\x14\\xe2\\x8d\\xe0\\xd8\\x13\\xb5\\xddn\\xe9\\xe3K\\xf0E\\xb1]' in any of [<Module tests/test_bytes.py>])

ERROR: not found: /home/user/test/tests/test_bytes.py::test_randbytes[\x00i\xb2\x08\x86\x99\x98"\x1bw<\xf4\x84[\xc4\xb9\x81\x9f\x00q\xaes\xce\xfd\xb4^\xbe\x12\x8e\xdcv\xff|\x06\x9d\x97]
(no name '/home/user/test/tests/test_bytes.py::test_randbytes[\\x00i\\xb2\\x08\\x86\\x99\\x98"\\x1bw<\\xf4\\x84[\\xc4\\xb9\\x81\\x9f\\x00q\\xaes\\xce\\xfd\\xb4^\\xbe\\x12\\x8e\\xdcv\\xff|\\x06\\x9d\\x97]' in any of [<Module tests/test_bytes.py>])

Finished running tests!

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Dec 15, 2023
@eleanorjboyd
Copy link
Member

ah I see now, we have been having some issues related to running pytest parameterized tests that generate different names for each run. The extension is looking for a given name but as the random number generates a new one, the name changes and it struggles to find what test to run. Im in the process of investigating, Ill update here with the results or another issue to reference. Thanks

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Dec 15, 2023
@eleanorjboyd eleanorjboyd removed the info-needed Issue requires more information from poster label Dec 18, 2023
@eleanorjboyd
Copy link
Member

Hi! Gave this some investigation. It seems right now that running parameterize tests with renamed tests is not feasible. How we handle tests is via referencing all the test items and their IDs in order to handle UI like run, the pass or fail checkmarks etc. Without a stable id we can use to reference these items, we cannot run them from our UI. If the IDs were going to be the same random number every time it would work. So for example if you generate a set of random numbers, save these as constants then use that set of numbers (which will be stable between runs) for every run it will work but right now test ids which change between runs cannot be supported. Thanks

@eleanorjboyd eleanorjboyd closed this as not planned Won't fix, can't repro, duplicate, stale Dec 18, 2023
@github-actions github-actions bot added the info-needed Issue requires more information from poster label Dec 18, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 18, 2024
@eleanorjboyd eleanorjboyd changed the title Pytest fails in VSCode but passes on command line pytest parameterized tests with randomly generate names Mar 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-testing info-needed Issue requires more information from poster triage-needed Needs assignment to the proper sub-team
Projects
None yet
Development

No branches or pull requests

3 participants