Running debugger under conda fails to load _sqlite3

[Astlaan]

Issue Type: Bug

Hello,

After editing some documents in a repository, staging the changes, adding a commit message, and then trying to commit, I get an error with the following Command output.

> git -c user.useConfigOnly=true commit --quiet --allow-empty-message --file -
Traceback (most recent call last):
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pre_commit\__main__.py", line 1, in <module>
    from pre_commit.main import main
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pre_commit\main.py", line 13, in <module>
    from pre_commit.commands.autoupdate import autoupdate
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pre_commit\commands\autoupdate.py", line 20, in <module>
    from pre_commit.store import Store
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pre_commit\store.py", line 4, in <module>
    import sqlite3
  File "C:\Users\me\Anaconda3\envs\myenv\lib\sqlite3\__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "C:\Users\me\Anaconda3\envs\myenv\lib\sqlite3\dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: DLL load failed while importing _sqlite3: The specified module could not be found.


> git -c user.useConfigOnly=true commit --quiet --allow-empty-message --file -
Traceback (most recent call last):
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pre_commit\__main__.py", line 1, in <module>
    from pre_commit.main import main
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pre_commit\main.py", line 13, in <module>
    from pre_commit.commands.autoupdate import autoupdate
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pre_commit\commands\autoupdate.py", line 20, in <module>
    from pre_commit.store import Store
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pre_commit\store.py", line 4, in <module>
    import sqlite3
  File "C:\Users\me\Anaconda3\envs\myenv\lib\sqlite3\__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "C:\Users\me\Anaconda3\envs\myenv\lib\sqlite3\dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: DLL load failed while importing _sqlite3: The specified module could not be found.

I did try importing those same python modules in a python session in a terminal (in the same anaconda environment) and I did not get any issue.

As far as I can tell, this is only happening one of the two workspaces I normally use.

VS Code version: Code 1.63.2 (899d46d82c4c95423fb7e10e68eba52050e30ba3, 2021-12-15T09:40:02.816Z)
OS version: Windows_NT x64 10.0.19042
Restricted Mode: No

System Info

Item	Value
CPUs	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz (8 x 1690)
GPU Status	2d_canvas: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: enabled opengl: enabled_on rasterization: enabled skia_renderer: enabled_on video_decode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	15.73GB (5.55GB free)
Process Argv	--crash-reporter-id c4587ea1-65de-4590-a2c4-cb48d86b4f8b
Screen Reader	no
VM	0%

Extensions (15)

Extension	Author (truncated)	Version
languagetool-linter	dav	0.19.0
typescript-notebook	don	2.0.6
gitlens	eam	11.7.0
gitlab-workflow	Git	3.40.1
todo-tree	Gru	0.0.215
gitlab-mr	jas	1.4.3
git-graph	mhu	1.30.0
python	ms-	2021.12.1559732655
vscode-pylance	ms-	2022.1.1
jupyter	ms-	2021.11.1001550889
jupyter-keymap	ms-	1.0.0
jupyter-renderers	ms-	1.0.4
powershell	ms-	2021.12.0
rust	rus	0.7.8
rewrap	stk	1.15.4

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythontb:30283811
pythonvspyt551:30345470
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
pythondataviewer:30285071
vscod805:30301674
pythonvspyt200:30340761
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
vsc1dst:30421291
pythonvs932:30410667
vscop804:30404766
vscop940:30404999
vsrem710:30416614
vsccppwtstudy:30421978

[lszomoru]

@Astlaan, are you able to reproduce the same behaviour if you start code using code --disable-extensions? Do these two workspaces have any git hooks installed that are being executed when you try to commit your code? Are you able to commit from the terminal?

[Astlaan]

@Astlaan, are you able to reproduce the same behaviour if you start code using code --disable-extensions? Do these two workspaces have any git hooks installed that are being executed when you try to commit your code? Are you able to commit from the terminal?

@lszomoru

• yes, the same behaviour happens if I start code with code --disable-extensions.
• I am able to commit from the terminal, in the same anaconda env.
• Yes, I am using a git hook. I tried running pre-commit uninstalled and it started working fine.

However, what is weird is that commiting in the terminal always worked, even with the pre-commit enabled.

The git hook file is the one below:

"pre-commit"

#!/usr/bin/env bash
# File generated by pre-commit: https://pre-commit.com
# ID: 138fd403232d2ddd5efb44317e38bf03

# start templated
INSTALL_PYTHON='C:\Users\me\Anaconda3\envs\myenv\python.exe'
ARGS=(hook-impl --config=.pre-commit-config.yaml --hook-type=pre-commit)
# end templated

HERE="$(cd "$(dirname "$0")" && pwd)"
ARGS+=(--hook-dir "$HERE" -- "$@")

if [ -x "$INSTALL_PYTHON" ]; then
    exec "$INSTALL_PYTHON" -mpre_commit "${ARGS[@]}"
elif command -v pre-commit > /dev/null; then
    exec pre-commit "${ARGS[@]}"
else
    echo '`pre-commit` not found.  Did you forget to activate your virtualenv?' 1>&2
    exit 1
fi

[Astlaan]

@lszomoru

The same thing appears to happen when I try to Debug Tests (pytest):

Traceback (most recent call last):
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Users\me\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy\__main__.py", line 45, in <module>
    cli.main()
  File "c:\Users\me\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main
    run()
  File "c:\Users\me\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\me\Anaconda3\envs\myenv\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Users\me\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\testlauncher.py", line 44, in <module>
    run(cwd, testRunner, args)
  File "c:\Users\me\.vscode\extensions\ms-python.python-2021.12.1559732655\pythonFiles\testlauncher.py", line 36, in run
    pytest.main(args)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\_pytest\config\__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\_pytest\config\__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pluggy\_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\_pytest\helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pluggy\_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\_pytest\config\__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\_pytest\config\__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\_pytest\config\__init__.py", line 1172, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pluggy\_manager.py", line 287, in load_setuptools_entrypoints
    plugin = ep.load()
  File "C:\Users\me\Anaconda3\envs\myenv\lib\importlib\metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "C:\Users\me\Anaconda3\envs\myenv\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\_pytest\assertion\rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\pytest_cov\plugin.py", line 6, in <module>
    import coverage
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\coverage\__init__.py", line 15, in <module>
    from coverage.control import Coverage, process_startup
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\coverage\control.py", line 21, in <module>
    from coverage.data import CoverageData, combine_parallel_data
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\coverage\data.py", line 18, in <module>
    from coverage.sqldata import CoverageData
  File "C:\Users\me\Anaconda3\envs\myenv\lib\site-packages\coverage\sqldata.py", line 18, in <module>
    import sqlite3
  File "C:\Users\me\Anaconda3\envs\myenv\lib\sqlite3\__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "C:\Users\me\Anaconda3\envs\myenv\lib\sqlite3\dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: DLL load failed while importing _sqlite3: The specified module could not be found.

[Astlaan]

@lszomoru any idea on how to solve this without having to deactivate the hook? :)

[Astlaan]

Any news?

[lszomoru]

@Astlaan, sorry for not getting back to you on this until now. Unfortunately, I am not a python expert, but my suspicion is that this is some kind of a difference between the "environment variables" that are being used by VS Code, and the Terminal given that everything workspace expected in the terminal but not when using VS Code. Is there a mechanism to trace out the the way that python resolves/loads dependencies?

[lszomoru]

@brettcannon, could someone from the python provide some guidance given that the error to load _sqlite3 also occurs when the user is trying to debug python tests? Thank you!

[brettcannon]

It's quite possible a side-effect of using a conda environment for running pre-commit. Conda environments typically need to be activated and if the git hook isn't doing that (e.g. via conda run) then that could cause conda to not find modules.

Otherwise I can't debug without more context (e.g. what version of the extension, what was provided on the command-line, etc.).

[lszomoru]

Thanks @brettcannon. Based on this comment, it looks like the same error is being shown when the user is trying to debug their tests. That scenario does not involve git at all, and it might have the same root cause. Could we approve the troubleshooting from that angle? Thanks!

[brettcannon]

@lszomoru I've gone ahead and transferred the issue to the vscode-python repo.

@Astlaan please let us know if you are still having this issue as we have completely reworked our conda support since you last reported anything.

[Astlaan]

Hi @brettcannon .

Now the issue I was having got replaced by this error message:

This is the console output:

> git -c user.useConfigOnly=true commit --quiet --allow-empty-message --file -
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

[brettcannon]

That's due to git not finding python anywhere as you probably don't have python on PATH if you're using conda. So that isn't due to anything in the Python extension and more about how git is being run and how your system is set up.

@lszomoru do you want this moved back to the vscode repo since this more about git and how it executes commands?

[Astlaan]

@brettcannon

Something I forgot to test was whether the pytest debugger was working, but it seems not:

Traceback (most recent call last):
  File "C:\Users\me\Anaconda3\envs\project\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\me\Anaconda3\envs\project\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Users\me\.vscode\extensions\ms-python.python-2022.4.1\pythonFiles\lib\python\debugpy\__main__.py", line 45, in <module>
    cli.main()
  File "c:\Users\me\.vscode\extensions\ms-python.python-2022.4.1\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main
    run()
  File "c:\Users\me\.vscode\extensions\ms-python.python-2022.4.1\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "C:\Users\me\Anaconda3\envs\project\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\me\Anaconda3\envs\project\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\me\Anaconda3\envs\project\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\Users\me\.vscode\extensions\ms-python.python-2022.4.1\pythonFiles\testlauncher.py", line 44, in <module>
    run(cwd, testRunner, args)
  File "c:\Users\me\.vscode\extensions\ms-python.python-2022.4.1\pythonFiles\testlauncher.py", line 36, in run
    pytest.main(args)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\_pytest\config\__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\_pytest\config\__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\pluggy\_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\_pytest\helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\pluggy\_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\_pytest\config\__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\_pytest\config\__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\_pytest\config\__init__.py", line 1172, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\pluggy\_manager.py", line 287, in load_setuptools_entrypoints
    plugin = ep.load()
  File "C:\Users\me\Anaconda3\envs\project\lib\importlib\metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "C:\Users\me\Anaconda3\envs\project\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\_pytest\assertion\rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\pytest_cov\plugin.py", line 6, in <module>
    import coverage
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\coverage\__init__.py", line 15, in <module>
    from coverage.control import Coverage, process_startup
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\coverage\control.py", line 21, in <module>
    from coverage.data import CoverageData, combine_parallel_data
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\coverage\data.py", line 18, in <module>
    from coverage.sqldata import CoverageData
  File "C:\Users\me\Anaconda3\envs\project\lib\site-packages\coverage\sqldata.py", line 18, in <module>
    import sqlite3
  File "C:\Users\me\Anaconda3\envs\project\lib\sqlite3\__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "C:\Users\me\Anaconda3\envs\project\lib\sqlite3\dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: DLL load failed while importing _sqlite3: The specified module could not be found.

To be clear, this happens when I use the following function:

[karrtikr]

@Astlaan This might be that when run in VS Code it is not running in a activated environment. Can you try this?

1. Anaconda prompt and cd to your workspace.
2. Activate the environment using conda activate
3. Start VS Code using code .

related #10626

[Astlaan]

@karrtikr

Thanks.

Indeed the previous error doesn't appear to pop up anymore.

I'm facing another possibly weird behaviour, but not sure if this is an issue from VS Code.

If I try "debug test", initially I get the debugger control buttons, as expected.

However, after running it seems the debugger quits and I am not left in a debugger session.

In contrast, if I run the equivalent bash command (with --pdb in order to debug the test):

pytest -k test_identical_pulses_compile --pdb

I am left with a running debugging session, as expected:

>       import jkjdfg
E       ModuleNotFoundError: No module named 'jkjdfg'

tests\scheduler\backends\test_qblox_backend.py:822: ModuleNotFoundError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 
> c:\users\user\desktop\qblox\code\quantify-scheduler\tests\scheduler\backends\test_qblox_backend.py(822)test_identical_pulses_compile()       
-> import jkjdfg
(Pdb)

[karrtikr]

@Astlaan, sadly there is an issue with conda run when running it from a terminal that already has an activated environment (conda/conda#11305, conda/conda#11174) due to which conda ends up using the base environment python instead, and hence results in errors such as "No module named 'jkjdfg'". So unfortunately there isn't much to do to be done here atm.

We're looking to activate conda environments in a different way with #11039, which should also help solve this issue, closing in favor of that.

As a workaround, you can use extension version which predates 2021.12.0 (9 December 2021), where we do not use conda run, that should not result in the module not found error.