Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conda Environment identification and renaming #24247

Closed
Breeze-Hu opened this issue Oct 4, 2024 · 22 comments · Fixed by microsoft/python-environment-tools#164
Closed

Conda Environment identification and renaming #24247

Breeze-Hu opened this issue Oct 4, 2024 · 22 comments · Fixed by microsoft/python-environment-tools#164
Assignees
Labels
area-native-locator Issues with Rust based python locator bug Issue identified by VS Code Team member as probable bug triage-needed Needs assignment to the proper sub-team verified Verification succeeded
Milestone

Comments

@Breeze-Hu
Copy link

Does this issue occur when all extensions are disabled?: Yes/No
Yes.

  • VS Code Version:
    版本: 1.94.0 (user setup)
    提交: d78a74bcdfad14d5d3b1b782f87255d802b57511
    日期: 2024-10-02T13:08:12.626Z
    Electron: 30.5.1
    ElectronBuildId: 10262041
    Chromium: 124.0.6367.243
    Node.js: 20.16.0
    V8: 12.4.254.20-electron.0
    OS: Windows_NT x64 10.0.22631

  • OS Version:
    Windows 11 23H2

Something unexpected seems to be happening when vscode recognizes conda virtual environments (WSL: Ubuntu-22.04). Rename the virtual environment with the name ’learn‘ to ’base‘. The previous 'base' environment, now named 'miniconda3'. While subsequent programs run fine, opening it for the first time caused some disturbances.

It's possible that the most recent update caused the problem, which shouldn't have happened before...

image

image

@vs-code-engineering vs-code-engineering bot added the triage-needed Needs assignment to the proper sub-team label Oct 4, 2024
@TylerLeonhardt TylerLeonhardt transferred this issue from microsoft/vscode Oct 4, 2024
@TylerLeonhardt TylerLeonhardt removed their assignment Oct 4, 2024
@karthiknadig
Copy link
Member

@Breeze-Hu Can you enable trace logging (Set Log level) and share the output from Python Locator? Please share all of it.

image

image

@karthiknadig karthiknadig self-assigned this Oct 4, 2024
@github-actions github-actions bot added the info-needed Issue requires more information from poster label Oct 4, 2024
@karthiknadig
Copy link
Member

@DonJayamanne I think this is coming from pet. Can you take a look?

@Breeze-Hu
Copy link
Author

@karthiknadig Hi, I'm not quite sure if this is the part of the message you're referring to. python locator provides these below, and I'm sure the issue still exists at this point.

2024-10-05 09:20:49.665 [info] Starting Python Locator /home/yu/.vscode-server/extensions/ms-python.python-2024.17.2024100401-linux-x64/python-env-tools/bin/pet server
2024-10-05 09:20:49.686 [info] Discovered manager: (Conda) /home/yu/miniconda3/bin/conda
2024-10-05 09:20:49.687 [info] Discovered env: /home/yu/miniconda3/bin/python
2024-10-05 09:20:49.687 [info] Discovered env: /bin/python3
2024-10-05 09:20:49.687 [info] Discovered env: /usr/bin/python3
2024-10-05 09:20:49.687 [info] Discovered manager: (Conda) /home/yu/miniconda3/envs/learn/bin/conda
2024-10-05 09:20:49.687 [info] Discovered env: /home/yu/miniconda3/envs/learn/bin/python
2024-10-05 09:20:50.846 [info] Discovered env: /home/yu/miniconda3/envs/cupy/bin/python
2024-10-05 09:20:50.846 [info] Discovered env: /home/yu/miniconda3/envs/requests/bin/python
2024-10-05 09:20:50.847 [info] Discovered env: /home/yu/miniconda3/envs/learn2/bin/python
2024-10-05 09:20:50.847 [info] Discovered env: /home/yu/miniconda3/envs/ML/bin/python
2024-10-05 09:20:50.848 [info] Discovered env: /home/yu/miniconda3/envs/xgboost/bin/python
2024-10-05 09:20:50.848 [info] Locator Conda took 1.697821648s
2024-10-05 09:20:50.848 [info] Locator Homebrew took 484.4µs
2024-10-05 09:20:50.848 [info] Locator LinuxGlobal took 14.724107ms
2024-10-05 09:20:50.848 [info] Locator PipEnv took 23.201µs
2024-10-05 09:20:50.848 [info] Locator Poetry took 17.375808ms
2024-10-05 09:20:50.848 [info] Locator PyEnv took 17.347908ms
2024-10-05 09:20:50.848 [info] Locator Venv took 140.3µs
2024-10-05 09:20:50.848 [info] Locator VirtualEnv took 18.3µs
2024-10-05 09:20:50.848 [info] Locator VirtualEnvWrapper took 10.4µs
2024-10-05 09:20:50.848 [info] Locator GlobalVirtualEnvs took 422.201µs
2024-10-05 09:20:50.848 [info] Locator Locators took 1.698151948s
2024-10-05 09:20:50.848 [info] Locator Path took 146.825065ms
2024-10-05 09:20:50.848 [info] Locator Workspaces took 800ns
2024-10-05 09:20:50.848 [info] Refresh completed in 1698ms

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Oct 5, 2024
@karthiknadig
Copy link
Member

@Breeze-Hu That is correct output. Can you do one more thing? Run that executable in the Output from a terminal and share the output:

/home/yu/.vscode-server/extensions/ms-python.python-2024.17.2024100401-linux-x64/python-env-tools/bin/pet

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Oct 5, 2024
@Breeze-Hu
Copy link
Author

@karthiknadig Hope this helps in solving the problem, the last result returned by the command you mentioned is as follows:

Manager (Conda)
   Executable  : /home/yu/miniconda3/bin/conda
   Version     : 24.9.1

Environment (Conda)
   Name        : base
   Executable  : /home/yu/miniconda3/bin/python
   Version     : 3.12.2
   Prefix      : /home/yu/miniconda3
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/bin/python"
               : "/home/yu/miniconda3/bin/python3"
               : "/home/yu/miniconda3/bin/python3.1"
               : "/home/yu/miniconda3/bin/python3.12"

Environment (LinuxGlobal)
   Executable  : /bin/python3
   Version     : 3.10.12.final.0
   Prefix      : /usr
   Architecture: x64
   Symlinks    : "/bin/python3"
               : "/bin/python3.10"

Environment (LinuxGlobal)
   Executable  : /usr/bin/python3
   Version     : 3.10.12.final.0
   Prefix      : /usr
   Architecture: x64
   Symlinks    : "/usr/bin/python3"
               : "/usr/bin/python3.10"

Manager (Conda)
   Executable  : /home/yu/miniconda3/envs/learn/bin/conda
   Version     : 24.9.1

Environment (Conda)
   Name        : base
   Executable  : /home/yu/miniconda3/envs/learn/bin/python
   Version     : 3.11.9
   Prefix      : /home/yu/miniconda3/envs/learn
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/envs/learn/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/learn/bin/python"
               : "/home/yu/miniconda3/envs/learn/bin/python3"
               : "/home/yu/miniconda3/envs/learn/bin/python3.1"
               : "/home/yu/miniconda3/envs/learn/bin/python3.11"

Environment (Conda)
   Name        : requests
   Executable  : /home/yu/miniconda3/envs/requests/bin/python
   Version     : 3.11.9
   Prefix      : /home/yu/miniconda3/envs/requests
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/requests/bin/python"
               : "/home/yu/miniconda3/envs/requests/bin/python3"
               : "/home/yu/miniconda3/envs/requests/bin/python3.1"
               : "/home/yu/miniconda3/envs/requests/bin/python3.11"

Environment (Conda)
   Name        : learn2
   Executable  : /home/yu/miniconda3/envs/learn2/bin/python
   Version     : 3.12.6
   Prefix      : /home/yu/miniconda3/envs/learn2
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/learn2/bin/python"
               : "/home/yu/miniconda3/envs/learn2/bin/python3"
               : "/home/yu/miniconda3/envs/learn2/bin/python3.1"
               : "/home/yu/miniconda3/envs/learn2/bin/python3.12"

Environment (Conda)
   Name        : cupy
   Executable  : /home/yu/miniconda3/envs/cupy/bin/python
   Version     : 3.11.10
   Prefix      : /home/yu/miniconda3/envs/cupy
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/cupy/bin/python"
               : "/home/yu/miniconda3/envs/cupy/bin/python3"
               : "/home/yu/miniconda3/envs/cupy/bin/python3.1"
               : "/home/yu/miniconda3/envs/cupy/bin/python3.11"

Environment (Conda)
   Name        : xgboost
   Executable  : /home/yu/miniconda3/envs/xgboost/bin/python
   Version     : 3.11.9
   Prefix      : /home/yu/miniconda3/envs/xgboost
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/xgboost/bin/python"
               : "/home/yu/miniconda3/envs/xgboost/bin/python3"
               : "/home/yu/miniconda3/envs/xgboost/bin/python3.1"
               : "/home/yu/miniconda3/envs/xgboost/bin/python3.11"

Environment (Conda)
   Name        : ML
   Executable  : /home/yu/miniconda3/envs/ML/bin/python
   Version     : 3.11.9
   Prefix      : /home/yu/miniconda3/envs/ML
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/ML/bin/python"
               : "/home/yu/miniconda3/envs/ML/bin/python3"
               : "/home/yu/miniconda3/envs/ML/bin/python3.1"
               : "/home/yu/miniconda3/envs/ML/bin/python3.11"


Breakdown by each locator:
--------------------------
Conda                : 1.49119364s
Homebrew             : 83.8µs
LinuxGlobal          : 68.099005ms
PipEnv               : 700ns
Poetry               : 15.389002ms
PyEnv                : 15.977802ms
Venv                 : 800ns
VirtualEnv           : 300ns
VirtualEnvWrapper    : 300ns

Breakdown for finding Environments:
-----------------------------------
GlobalVirtualEnvs    : 176.9µs
Locators             : 1.49150614s
Path                 : 151.187013ms
Workspaces           : 20.040102ms

Managers:
---------
Conda                : 2

Environments (9):
------------------
Conda                : 7
LinuxGlobal          : 2

Completed in 1509ms

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Oct 6, 2024
@karthiknadig
Copy link
Member

We are still trying to diagnose the issue. The log you provided will help with that.

@DonJayamanne this looks like pet is classifying named environment as base.

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Oct 6, 2024
@Breeze-Hu
Copy link
Author

Hope it goes well for you all. Thank you very much!

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Oct 6, 2024
@DonJayamanne
Copy link

DonJayamanne commented Oct 6, 2024

@Breeze-Hu Please can you list the files and directories inside this folder /home/yu/miniconda3/envs/learn
Or take a screenshot of the contents of the above directory.

It seems to have a condabin and/or a envs directory, which is unusual. & seems to be causing this false identification.

@DonJayamanne DonJayamanne self-assigned this Oct 6, 2024
@DonJayamanne DonJayamanne added bug Issue identified by VS Code Team member as probable bug info-needed Issue requires more information from poster area-native-locator Issues with Rust based python locator and removed triage-needed Needs assignment to the proper sub-team labels Oct 6, 2024
@DonJayamanne DonJayamanne added this to the October 2024 milestone Oct 6, 2024
@Breeze-Hu
Copy link
Author

Okay! The directory contains these files. There are indeed files under condabin and envs as you mentioned. The condabin file opens like this, opening the envs file seems to be empty.
Hope this information is useful! @DonJayamanne

image
image
image
image

@DonJayamanne
Copy link

Thanks, it looks like somehow you have configured the miniconda3/envs/learn/envs folder to be the location for conda environments.
Not sure how this happened, envs/.conda_envs_dir_test should not exist in a conda environment, except for the base environment, (looking at the source of conda, this shouldn't happen). @karthiknadig /cc
I'm guessing the same applies to condabin as well, though I'm not sure about that, either way, this isn't a valid conda environment setup. I'm assuming something went wrong somewhere resulting in these files/folders getting created.

I'll update the discovery tool to exclude such folders,
More info on .conda_envs_dir_test can be found in the source here https://github.com/search?q=repo%3Aconda%2Fconda%20conda_envs_dir_test&type=code

@Breeze-Hu
Copy link
Author

Thank you developers for your contributions! Expect this issue to be resolved in a subsequent update!

@DonJayamanne
Copy link

Yes, here's the PR microsoft/python-environment-tools#164
Once we get this merged and have a pre-release version of Python extension you can then verify that.
Will ping you when its ready to be tested.

@DonJayamanne DonJayamanne removed the info-needed Issue requires more information from poster label Oct 6, 2024
karthiknadig pushed a commit to karthiknadig/python-environment-tools that referenced this issue Oct 8, 2024
karthiknadig added a commit to microsoft/python-environment-tools that referenced this issue Oct 8, 2024
Cherry picking fixes into release branch.
Fixes microsoft/vscode-python#24247

Co-authored-by: Don Jayamanne <don.jayamanne@outlook.com>
@amunger amunger added the author-verification-requested Issues potentially verifiable by issue author label Oct 23, 2024
@amunger
Copy link

amunger commented Oct 23, 2024

@Breeze-Hu can you try to verify the fix on the latest pre-release version of the python extension

@Breeze-Hu
Copy link
Author

I apologize for bringing bad news. The problem is still there. I tried restarting VScode and WSL, neither of which resolved it.

Image

@DonJayamanne
Copy link

Please can you share the same output, as you did here #24247 (comment)
@karthiknadig Does pre-release version of Python extension automatically get the latest changes from PET?

@Breeze-Hu
Copy link
Author

Absolutely! I feel very close to solving the problem, although I do not understand it very well.

Manager (Conda)
   Executable  : /home/yu/miniconda3/bin/conda
   Version     : 24.9.2

Environment (Conda)
   Name        : base
   Executable  : /home/yu/miniconda3/bin/python
   Version     : 3.12.2
   Prefix      : /home/yu/miniconda3
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/bin/python"
               : "/home/yu/miniconda3/bin/python3"
               : "/home/yu/miniconda3/bin/python3.1"
               : "/home/yu/miniconda3/bin/python3.12"

Environment (LinuxGlobal)
   Executable  : /bin/python3
   Version     : 3.10.12.final.0
   Prefix      : /usr
   Architecture: x64
   Symlinks    : "/bin/python3"
               : "/bin/python3.10"

Environment (LinuxGlobal)
   Executable  : /usr/bin/python3
   Version     : 3.10.12.final.0
   Prefix      : /usr
   Architecture: x64
   Symlinks    : "/usr/bin/python3"
               : "/usr/bin/python3.10"

Environment (Conda)
   Name        : learn
   Executable  : /home/yu/miniconda3/envs/learn/bin/python
   Version     : 3.11.9
   Prefix      : /home/yu/miniconda3/envs/learn
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/learn/bin/python"
               : "/home/yu/miniconda3/envs/learn/bin/python3"
               : "/home/yu/miniconda3/envs/learn/bin/python3.1"
               : "/home/yu/miniconda3/envs/learn/bin/python3.11"

Environment (Conda)
   Name        : requests
   Executable  : /home/yu/miniconda3/envs/requests/bin/python
   Version     : 3.11.9
   Prefix      : /home/yu/miniconda3/envs/requests
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/requests/bin/python"
               : "/home/yu/miniconda3/envs/requests/bin/python3"
               : "/home/yu/miniconda3/envs/requests/bin/python3.1"
               : "/home/yu/miniconda3/envs/requests/bin/python3.11"

Environment (Conda)
   Name        : learn2
   Executable  : /home/yu/miniconda3/envs/learn2/bin/python
   Version     : 3.12.6
   Prefix      : /home/yu/miniconda3/envs/learn2
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/learn2/bin/python"
               : "/home/yu/miniconda3/envs/learn2/bin/python3"
               : "/home/yu/miniconda3/envs/learn2/bin/python3.1"
               : "/home/yu/miniconda3/envs/learn2/bin/python3.12"

Environment (Conda)
   Name        : cupy
   Executable  : /home/yu/miniconda3/envs/cupy/bin/python
   Version     : 3.11.10
   Prefix      : /home/yu/miniconda3/envs/cupy
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/cupy/bin/python"
               : "/home/yu/miniconda3/envs/cupy/bin/python3"
               : "/home/yu/miniconda3/envs/cupy/bin/python3.1"
               : "/home/yu/miniconda3/envs/cupy/bin/python3.11"

Environment (Conda)
   Name        : ML
   Executable  : /home/yu/miniconda3/envs/ML/bin/python
   Version     : 3.11.9
   Prefix      : /home/yu/miniconda3/envs/ML
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/ML/bin/python"
               : "/home/yu/miniconda3/envs/ML/bin/python3"
               : "/home/yu/miniconda3/envs/ML/bin/python3.1"
               : "/home/yu/miniconda3/envs/ML/bin/python3.11"

Environment (Conda)
   Name        : xgboost
   Executable  : /home/yu/miniconda3/envs/xgboost/bin/python
   Version     : 3.11.9
   Prefix      : /home/yu/miniconda3/envs/xgboost
   Architecture: x64
   Manager     : Conda, /home/yu/miniconda3/bin/conda
   Symlinks    : "/home/yu/miniconda3/envs/xgboost/bin/python"
               : "/home/yu/miniconda3/envs/xgboost/bin/python3"
               : "/home/yu/miniconda3/envs/xgboost/bin/python3.1"
               : "/home/yu/miniconda3/envs/xgboost/bin/python3.11"


Breakdown by each locator:
--------------------------
Conda                : 1.436896797s
Homebrew             : 21.3µs
LinuxGlobal          : 38.645392ms
PipEnv               : 300ns
Poetry               : 18.496696ms
PyEnv                : 18.233796ms
Venv                 : 200ns
VirtualEnv           : 300ns
VirtualEnvWrapper    : 200ns

Breakdown for finding Environments:
-----------------------------------
GlobalVirtualEnvs    : 240.8µs
Locators             : 1.437103997s
Path                 : 154.838567ms
Workspaces           : 15.589097ms

Managers:
---------
Conda                : 1

Environments (9):
------------------
Conda                : 7
LinuxGlobal          : 2

Completed in 1459ms

@DonJayamanne
Copy link

@karthiknadig looks like PET is returning the right information, The learn Conda is named learn.

@DonJayamanne DonJayamanne reopened this Oct 24, 2024
@DonJayamanne DonJayamanne removed their assignment Oct 24, 2024
@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Oct 24, 2024
@DonJayamanne DonJayamanne removed the author-verification-requested Issues potentially verifiable by issue author label Oct 24, 2024
@karthiknadig
Copy link
Member

@Breeze-Hu Can you try these two things?

  1. Ensure python.locator setting is set to native.
  2. Run Python: Clear Cache and reload command form the command palette.

After it reloads see if you get the same results.

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Oct 24, 2024
@Breeze-Hu
Copy link
Author

@Breeze-Hu Can you try these two things?

  1. Ensure python.locator setting is set to native.
  2. Run Python: Clear Cache and reload command form the command palette.

After it reloads see if you get the same results.

When I set the python.locator setting is set to native, the environment recognition is already correct!

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Oct 24, 2024
@Breeze-Hu
Copy link
Author

A brief test shows that it should be a problem with python.locator. When I switched it to js the problem appeared again.

@karthiknadig
Copy link
Member

@Breeze-Hu js is the old way, native is the new way of discovering environment. So you can leave it to native.

If it is working with native we can close this issue, we plan on deprecating js eventually.

@Breeze-Hu
Copy link
Author

I think this issue has been well addressed and it is good time to close the issue.
Thanks to all the developers for their efforts!

@amunger amunger added the verified Verification succeeded label Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-native-locator Issues with Rust based python locator bug Issue identified by VS Code Team member as probable bug triage-needed Needs assignment to the proper sub-team verified Verification succeeded
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants