Unable to discover Pixi environment

[eightysteele]

Type: Bug

I am connecting to a remote host (using Remote Development extension) that has a pixi project. I have the pre-release version of the MS Python extension. When I try to select a Python interpreter, none are available. I see this in the logs:

024-08-05 21:10:53.415 [info] Experiment 'pythonCreateEnvOnPipInstall' is active
2024-08-05 21:10:53.415 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-08-05 21:10:53.415 [info] Experiment 'pythonSurveyNotificationcf' is active
2024-08-05 21:10:53.415 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-08-05 21:10:53.415 [info] Experiment 'pythonTestAdapter' is active
2024-08-05 21:10:53.415 [info] Default formatter is set to null for workspace /home/eighty
2024-08-05 21:10:53.415 [info] Python interpreter path: ./b3d/.pixi/envs/default/bin/python
2024-08-05 21:10:53.470 [info] > pyenv which python
2024-08-05 21:10:53.470 [info] cwd: .
2024-08-05 21:10:53.559 [info] > pixi --version
2024-08-05 21:10:53.560 [error] [Error: spawn pixi ENOENT
	at ChildProcess._handle.onexit (node:internal/child_process:286:19)
	at onErrorNT (node:internal/child_process:484:16)
	at process.processTicksAndRejections (node:internal/process/task_queues:82:21)] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn pixi',
  path: 'pixi',
  spawnargs: [ '--version' ]
}
2024-08-05 21:10:53.560 [warning] could not find a pixi interpreter for the interpreter at /home/eighty/b3d/.pixi/envs/default/bin/python
2024-08-05 21:10:53.592 [info] > conda info --json
2024-08-05 21:10:54.743 [info] Starting Pylance language server.

Extension version: 2024.13.2024080501
VS Code version: Code 1.92.0 (b1c0a14de1414fcdaa400695b4db1c0799bc3124, 2024-07-31T23:26:45.634Z)
OS version: Linux x64 6.9.3-76060903-generic
Modes:
Remote OS version: Linux x64 6.5.0-1023-gcp

• Python version (& distribution if applicable, e.g. Anaconda): 3.12.4
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Pixi
• Value of the python.languageServer setting: Default

User Settings

languageServer: "Pylance"

Installed Extensions

Extension Name	Extension Id	Version
JavaScript Debugger	ms-vscode.js-debug	1.92.0
Jupyter	ms-toolsai.jupyter	2024.8.2024080201
Jupyter Cell Tags	ms-toolsai.vscode-jupyter-cell-tags	0.1.9
Jupyter Notebook Renderers	ms-toolsai.jupyter-renderers	1.0.19
Jupyter Slide Show	ms-toolsai.vscode-jupyter-slideshow	0.1.6
Pylance	ms-python.vscode-pylance	2024.7.1
Python	ms-python.python	2024.13.2024080501
Python Debugger	ms-python.debugpy	2024.10.0
Table Visualizer for JavaScript Profiles	ms-vscode.vscode-js-profile-table	1.0.9

System Info

Item	Value
CPUs	13th Gen Intel(R) Core(TM) i9-13900H (20 x 2586)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off webnn: disabled_off
Load (avg)	1, 1, 1
Memory (System)	31.19GB (16.89GB free)
Process Argv	--crash-reporter-id 01574559-2a39-4ee0-bc34-438099f06da6
Screen Reader	no
VM	0%
DESKTOP_SESSION	pop
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	pop
XDG_SESSION_TYPE	x11

Item	Value
Remote	SSH: aaron-b3d-test.us-west1-a.probcomp-caliban
OS	Linux x64 6.5.0-1023-gcp
CPUs	Intel(R) Xeon(R) CPU @ 2.20GHz (32 x 0)
Memory (System)	125.78GB (121.85GB free)
VM	0%

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscoreces:30445986
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
vscaat:30438848
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythongtdpath:30769146
welcomedialogc:30910334
pythonnoceb:30805159
asynctok:30898717
pythonregdiag2:30936856
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
f3je6385:31013174
pythoncenvpt:31062603
a69g1124:31058053
dvdeprecation:31068756
dwnewjupytercf:31046870
impr_priority:31102340
nativerepl1:31104043
refactort:31108082
ccplti:31103428
pythonrstrctxt:31103193
1iic9185:31097392
wkspc-onlycs-c:31106320

[eightysteele]

(Just to be clear, the pixi binary is available on the host and in the PATH)

[anthonykim1]

@eightysteele Thanks for filing this.
Does the entire extension crash on you/not load, or is the only symptom that you cannot see any available Python interpreters.

Also does this problem persist when you DONT have pixi for a Python project?

[eightysteele]

@anthonykim1 no crash, the only symptom is that I cannot see any available Python interpreters for my pixi project. In particular, I have two environments defined (cpu and gpu) and they don't show up.

Here's a repro....

I create a new Google Cloud VM (using gcloud) and then connect to it through vscode using the remote development extension.

From vscode terminal that's connected to my VM, I do the following:

conda deactivate
curl -fsSL https://pixi.sh/install.sh | bash
source ~/.bashrc 
pixi init pixi-py --pyproject
cd pixi-py/
mkdir pixi_py
touch pixi_py/__init__.py
conda deactivate
pixi add black
pixi add --pypi --feature test pytest
pixi add black --pypi
pixi add black[cli] --pypi
pixi install

Then I install the official MS Python extension (pre-release) and open the "output" window and select "Python". Here's what i see:

2024-08-06 15:37:21.517 [info] Experiment 'pythonCreateEnvOnPipInstall' is active
2024-08-06 15:37:21.517 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-08-06 15:37:21.517 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-08-06 15:37:21.517 [info] Experiment 'pythonTestAdapter' is active
2024-08-06 15:37:21.517 [info] Default formatter is set to null for workspace /home/b3d/pixi-py
2024-08-06 15:37:21.517 [info] Starting Environment refresh
2024-08-06 15:37:21.517 [info] > pixi --version
2024-08-06 15:37:21.517 [error] [Error: spawn pixi ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:286:19)
    at onErrorNT (node:internal/child_process:484:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn pixi',
  path: 'pixi',
  spawnargs: [ '--version' ]
}
2024-08-06 15:37:21.517 [info] > hatch env show --json
2024-08-06 15:37:21.517 [info] cwd: .
2024-08-06 15:37:21.517 [warning] Failed to check if /usr/local/bin/python3 is an executable [Error: ENOENT: no such file or directory, lstat '/usr/local/bin/python3'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'lstat',
  path: '/usr/local/bin/python3'
}
2024-08-06 15:37:21.528 [info] > /usr/local/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.12.2-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.12.2-linux-x64/python_files/interpreterInfo.py
2024-08-06 15:37:21.528 [info] Environments refresh paths discovered: 451 milliseconds
2024-08-06 15:37:21.530 [error] [Error: Command failed: /usr/local/bin/python3 -I /home/b3d/.vscode-server/extensions/ms-python.python-2024.12.2-linux-x64/python_files/get_output_via_markers.py /home/b3d/.vscode-server/extensions/ms-python.python-2024.12.2-linux-x64/python_files/interpreterInfo.py
/bin/sh: 1: /usr/local/bin/python3: not found

And then as a final thing, "Select Interpreter" doesn't show pixi environments (unsurprisingly, given the errors).

Hope that helps!

[anthonykim1]

@eightysteele Thanks much for detailed logs here.

I just merged in #23937 so hopefully that will clear things out for us. Error should be less aggressive but also more verbose and detailed. That being said, starting TOMORROW, can you please give the latest pre-release of Python extension a try and see if you get the same symptoms + output logs?

For your scenario, it seems more like the issue with discovery of pixi environment itself?

My follow up questions are:

1. Do other non-pixi environment show up in the "select interpreter"?
2. Does the problem persist when you are in non-remote scenario.

[anthonykim1]

I have additional PR here: #23942

[eightysteele]

@anthonykim1 thank you! is there anything i can do to help test out these changes?

[anthonykim1]

Hello @eightysteele Can you install the latest pre-release version of Python extension and see if you can repro the problem? I've merged in #23937 and #23942 will be merged with approval.

I would assume

2024-08-06 15:37:21.517 [error] [Error: spawn pixi ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:286:19)
    at onErrorNT (node:internal/child_process:484:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn pixi',
  path: 'pixi',
  spawnargs: [ '--version' ]

will still be there without #23942, but error should all go away when #23942 is merged too.

[eightysteele]

Hi @anthonykim1! In v2024.15.2024091301 (pre-release) when I open a Pixi project folder, it doesn't auto-detect the environments defined in .pixi/envs. Is that expected behavior? I can send a repro if helpful. Thanks!

[anthonykim1]

Hello @eightysteele Sorry that you are facing this :/

Was hoping that it would be auto-detected but we are seeing some issues with pixi discovery at the moment.
Have you tried manually entering the path to the pixi env?
#23975 (comment) might help you. Make sure you enter the relative path in respect to your workspace and not the absolute path for the reason of: #24042

Let me know how this goes! -Thanks

[hadim]

I am hitting the same issue where the local pixi default env is never detected and I must enter its path manually every time.

[hadim]

Could that be due to pixi introducing environments a few months ago?

In short there is no one single env anymore such as .pixi/env/bin/python but potentially many with one default one called default such as .pixi/envs/default/bin/python.

[anthonykim1]

@hadim when you enter the path for pixi, are you able to use that pixi environment? I'm curious of whether you are entering an absolute path, relative to workspace, etc

Could that be due to pixi introducing environments a few months ago?

This is quite possible. And also related: #24042

I'm curious to see if switching the native locator setting to be set to "js" help with automatically detecting pixi environment for you guys. @eightysteele @hadim

Thanks all for your support,

[hadim]

Switching to python locator js works for me (after restarting vscode).

[anthonykim1]

@hadim Working in a sense that your pixi environment is properly discovered, AND you were able to use that pixi environment, correct?

[hadim]

yes exactly. all the pixi's envs in the current workspace are correctly detected and they also all work.

[anthonykim1]

@hadim that helps a ton, thank you for verifying that.
That is expected behavior at the moment - the "native" locator doesn't support discovering pixi environment atm, so I would just leave your locator to "js" so that it could auto-detect your pixi environment. I'm hoping pixi discovery for native locator comes in the future, but can't guarantee you anything now.

I'm curious to see if the workaround works for @eightysteele too!