pylint extension-pkg-whitelist=cv2 not working only when through the extension (Anaconda)

[747]

Environment data

• VS Code version: 1.42.1
• Extension version (available under the Extensions sidebar): 2020.2.64397
• OS and version: Win 10 64 bit Home
• Python version (& distribution if applicable, e.g. Anaconda): 3.7.6 64-bit Anaconda
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda
• Relevant/affected Python packages and their versions:

pylint 2.4.4
astroid 2.3.3
Python 3.7.6 (default, Jan  8 2020, 20:23:39) [MSC v.1916 64 bit (AMD64)]

• Relevant/affected Python-related VS Code extensions and their versions: Code Runner 0.9.16
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version #3977): Jedi
• Value of the python.languageServer setting: Microsoft

Expected behaviour

If I set argument "python.linting.pylintArgs": ["--extension-pkg-whitelist=cv2"], the linter should look up members in the cv2 module.

Actual behaviour

Steps to reproduce:

1. Open a script that imports cv2 in VSCode.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

> ~\Anaconda3\envs\cv\python.exe -m pylint --extension-pkg-whitelist=cv2 --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text s:\shared\new_scripts\cvprocess.py
cwd: s:\shared\new_scripts
##########Linting Output - pylint##########
************* Module cvprocess
...
61,8,convention,invalid-name:Variable name "nh" doesn't conform to snake_case naming style
67,12,error,no-member:Module 'cv2' has no 'imread' member
68,22,error,no-member:Module 'cv2' has no 'threshold' member
68,35,error,no-member:Module 'cv2' has no 'imread' member
68,66,error,no-member:Module 'cv2' has no 'THRESH_BINARY' member
69,18,error,no-member:Module 'cv2' has no 'Canny' member
71,12,error,no-member:Module 'cv2' has no 'HoughLinesP' member
25,27,warning,unused-argument:Unused argument 'time'
...
-------------------------------------------------------------------

Your code has been rated at -5.48/10 (previous run: 2.86/10, -8.33)

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

(cv) S:\shared\new_scripts>%userprofile%\Anaconda3\envs\cv\python.exe -m pylint --extension-pkg-whitelist=cv2 --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text s:\shared\new_scripts\cvprocess.py
************* Module cvprocess
...
61,8,convention,invalid-name:Variable name "nh" doesn't conform to snake_case naming style
25,27,warning,unused-argument:Unused argument 'time'
...
-------------------------------------------------------------------
Your code has been rated at 2.86/10 (previous run: -5.48/10, +8.33)

This was originally posted as a pylint issue but they diverted me here. It is really weird that running the same exe (I hope) with the same arguments results in different outputs. My terminal is set to Anaconda Prompt in the workspace.

Similar issues I found: #99, #606, #1648, #1993, #2879, #3562, #3802, #5257

[karthiknadig]

@kimadeline It could be that when we are running pylint it is running in an unactivated environment. It is likely that pylint is not able to resolve cv2 at all, and it is not able to add it to whitelist.

[ychechik]

This solves the issue for me:
"python.linting.pylintArgs": ["--generate-members"]

[747]

@ychechik I'm afraid it's not the cure but crashing pylint by a nonexistent parameter, that obviously prevents from detecting any errors.
See: pylint-dev/pylint#2426 (comment)
But the solution they described doesn't work for me either.

[kimadeline]

Hi @747, thank you for reaching out and your detailed research!

Just to confirm what @karthiknadig said, does it work if you run %userprofile%\Anaconda3\envs\cv\python.exe -m pylint --extension-pkg-whitelist=cv2 in a terminal instance that doesn't have an activated environment?

[747]

@kimadeline Hi, I believe your comment contains an important keyword but I don't think I fully understand it. Namely, what should I do when I have to run something "in a terminal instance that doesn't have an activated environment"? Is this a Python term or Anaconda's?
I only have Anaconda installation in my machine, so when I switch to the other environment I just get:

3,0,error,import-error:Unable to import 'cv2'

Oh, I include my workspace settings (which were in the original issue).

		"python.pythonPath": "My\\Path\\to\\Anaconda3\\envs\\cv\\python.exe",
		"python.autoComplete.extraPaths": [
			"My\\Path\\to\\Anaconda3\\Lib\\site-packages"
		],
		"python.linting.pylintArgs": [
			"--generated-members=cv2.*",
			"--extension-pkg-whitelist=cv2"
		]

[kimadeline]

Environments are a Python concept, and conda/Anaconda have their own spin on it. From the conda documentation:

A conda environment is a directory that contains a specific collection of conda packages that you have installed. For example, you may have one environment with NumPy 1.7 and its dependencies, and another environment with NumPy 1.6 for legacy testing. If you change one environment, your other environments are not affected. You can easily activate or deactivate environments, which is how you switch between them.

There are good online resources explaining the difference between Python and conda environments, for example this one, but I disgress, let's get back to our issue 🙂

In the terminal output you copied:

(cv) S:\shared\new_scripts>%userprofile%\Anaconda3\envs\cv\python.exe -m pylint --extension-pkg-whitelist=cv2 --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text s:\shared\new_scripts\cvprocess.py
************* Module cvprocess
...
61,8,convention,invalid-name:Variable name "nh" doesn't conform to snake_case naming style
25,27,warning,unused-argument:Unused argument 'time'
...
-------------------------------------------------------------------
Your code has been rated at 2.86/10 (previous run: -5.48/10, +8.33)

The first line shows that the cv environment is activated inside your terminal, which gives your terminal access to the packages installed inside that environment.

However, when the extension runs commands on behalf of the user like linting or formatting, it doesn't do so inside an environment, which is what we're trying to replicate here.

👉 When you open a terminal, does the cv environment get activated automatically (does the prompt line start with (cv))? If so, you need to deactivate the environment for this terminal instance first by running conda deactivate. If after that you have the (base) environment activated, you need to run conda deactivate again, so that you end up with no environment prefix:

And from there, run the same command as what the extension was running, namely %userprofile%\Anaconda3\envs\cv\python.exe -m pylint --extension-pkg-whitelist=cv2.

[747]

@kimadeline Thank you for your kind instruction. So I have tried it:

(cv) S:\shared\new_scripts>conda deactivate
S:\shared\new_scripts>"%userprofile%\Anaconda3\envs\cv\python.exe" -m pylint --extension-pkg-whitelist=cv2 cvprocess.py
************* Module cvprocess
...
cvprocess.py:67:12: E1101: Module 'cv2' has no 'imread' member (no-member)
cvprocess.py:68:22: E1101: Module 'cv2' has no 'threshold' member (no-member)
cvprocess.py:68:35: E1101: Module 'cv2' has no 'imread' member (no-member)
cvprocess.py:68:66: E1101: Module 'cv2' has no 'THRESH_BINARY' member (no-member)
cvprocess.py:69:18: E1101: Module 'cv2' has no 'Canny' member (no-member)
cvprocess.py:71:12: E1101: Module 'cv2' has no 'HoughLinesP' member (no-member)
...
-------------------------------------------------------------------
Your code has been rated at -5.48/10 (previous run: 0.48/10, -5.95)

Now I can confirm that it outputs exactly same errors the extension does.

[kimadeline]

What happens if you add --generated-members=cv2.* to the command being run? It would look like "%userprofile%\Anaconda3\envs\cv\python.exe" -m pylint --generated-members=cv2.* --extension-pkg-whitelist=cv2 cvprocess.py.

[747]

It still seems to print the same errors.

(cv) S:\shared\new_scripts>conda deactivate

S:\shared\new_scripts>"%userprofile%\Anaconda3\envs\cv\python.exe" -m pylint --generated-members=cv2.* --extension-pkg-whitelist=cv2 cvprocess.py
************* Module cvprocess
...
cvprocess.py:67:12: E1101: Module 'cv2' has no 'imread' member (no-member)
cvprocess.py:68:22: E1101: Module 'cv2' has no 'threshold' member (no-member)
cvprocess.py:68:35: E1101: Module 'cv2' has no 'imread' member (no-member)
cvprocess.py:68:66: E1101: Module 'cv2' has no 'THRESH_BINARY' member (no-member)
cvprocess.py:69:18: E1101: Module 'cv2' has no 'Canny' member (no-member)
cvprocess.py:71:12: E1101: Module 'cv2' has no 'HoughLinesP' member (no-member)
...
--------------------------------------------------------------------
Your code has been rated at -5.48/10 (previous run: -5.48/10, +0.00)

[kimadeline]

Hmmm 🤔

I will try to replicate your issue, see if we can walk through this together. Which command(s) did you use to install OpenCV? Do you also have a sample repo or sample code I could use to reproduce your pylint issue?

Thanks!

[747]

If my record is complete, this is all I have done to set up the environment (using Anaconda Prompt, IIRC).

> conda create -n cv python=3.7 anaconda
> conda activate cv
> pip install opencv-python
> pip install opencv-contrib-python

[kimadeline]

Sorry I didn't get back to you sooner @747!

Can you try opening VS Code from inside a terminal (code .) without deactivating it?

[747]

Hi, do you mean opening VSCode from external terminal (prompt)?

I tried launching VSCode from Anaconda Prompt:

(cv) C:\WINDOWS\system32>cd /d S:\shared\new_scripts

(cv) S:\shared\new_scripts>code .

When I call my workspace inside the new VSCode, pylint complains no more about missing members 😮
So, is that the correct way to use VSCode with Anaconda?

[kimadeline]

Hurray it worked 🥳

Yes, you should be launching VS Code from the Anaconda prompt. On our side we should make it more explicit in the docs (microsoft/vscode-docs#3481), and we will work towards telling users directly while in the extension as well (#10626). You are more than welcome to upvote these 2 issues to bump up their visibility and priority.

Using conda run to run tools in the extension would most likely solve the pylint issue you reported originally, however it isn't stable enough yet for us to use it (#9490).

Closing in favour of microsoft/vscode-docs#3481 and #10626.

Thank you for your patience!

[747]

Thank you for all your time and support, otherwise I'd not ever realize that problem. I'm going to support those issues.