python / python/cpython

sys.executable can be a relative path contrary to documented behavior [3.11 regression]

Open
#154,160 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

When PATH contains a relative directory, sys.executable will also be a relative directory contrary to the current sys.executable documentation:

A string giving the absolute path of the executable binary for the Python interpreter, on systems where this makes sense. If Python is unable to retrieve the real path to its executable, sys.executable will be an empty string or None.

Context: I have a script that uses a custom virtualenv for dependencies that invokes itself in a different working directory. Because cwd is different, it relies on os.executable to be absolute. Minimal reproducer, set up python3 -m venv venv, then run PATH="venv/bin:$PATH" python3 script.py; using $PWD/venv/bin instead is a workaround for this issue.

import sys, subprocess; subprocess.run([sys.executable, "--version"], cwd="/tmp")

Alternative reproducer that reveals the issue:

$ python3 -m venv venv && PATH="venv/bin:$PATH" python3 -c 'import sys;print(sys.executable, sys.version.split()[0])'
<frozen site>:101: RuntimeWarning: Unexpected value in sys.prefix, expected /home/arch/venv, got venv
<frozen site>:101: RuntimeWarning: Unexpected value in sys.exec_prefix, expected /home/arch/venv, got venv
venv/bin/python3 3.14.6

This used to work in Python 3.10 and before, and is broken in Python 3.11 (2022) up to at least 3.14.6 and 3.15.0b3. I used this reproducer to confirm:

$ podman run --rm python:3.11-slim env PATH=usr/local/bin python3 -c 'import sys;print(sys.executable, sys.version.split()[0])'
usr/local/bin/python3 3.11.15

Another mild regression is the presence of an extra leading slash when using a relative path from the root directory:

$ podman run --rm python:3.15-rc-slim usr/local/bin/python3 -c 'import sys;print(sys.executable, sys.version.split()[0])'
//usr/local/bin/python3 3.15.0b3

$ podman run --rm python:3.10-slim usr/local/bin/python3 -c 'import sys;print(sys.executable, sys.version.split()[0])'
/usr/local/bin/python3 3.10.20

I suspect that both regressions have something to do with the refactoring done in https://bugs.python.org/issue45582 (#29041 by @zooba).
Somewhat related is #124241, that report is not about a relative path, but a modified argv[0]. Its current proposed fix to rely on /proc/self/exe (#145486 by @FFY00) could fix this issue, but has new regression risks.

CPython versions tested on:

3.15, 3.14, 3.13, 3.12, 3.11

Operating systems tested on:

Linux

Linked PRs
  • gh-154163

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the minimal Python and container reproducers in the issue, then read the refactoring from #29041 and the related proposals in #124241 and PR #145486. The linked PR #154163 indicates work is already underway; done should restore an absolute sys.executable for relative PATH entries and avoid the extra leading slash across the affected Python versions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.