python-poetry / python-poetry/poetry

`poetry run` can fail for scripts defined in `[project.scripts]` if PATH is too long

Open
#10,482 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

Description

On Windows, defining a script under [project.scripts] (importantly, not under [tool.poetry.scripts]), and invoking it with poetry run myscript while the PATH environment variable is near or over ~8192 characters, results in:

'myscript' is not recognized as an internal or external command,
operable program or batch file.

This is the bug described in https://github.com/python/cpython/issues/137254, caused by the following code: https://github.com/python-poetry/poetry/blob/b580e8aa4fbce53569420e7b42568dfd9e73519f/src/poetry/utils/env/base_env.py#L445-L446

Notes

While inspecting the Poetry source code, I noticed a few things.

Scripts defined in [tool.poetry.scripts] get special treatment, where they also go through base_env.execute, but the call is made with python -c <inline script> based on the entrypoint's definition, rather than actually calling myscript.cmd (see RunCommand.handle(), RunCommand.run_script()). Scripts defined in [project.scripts] are not taken into account when the check is made, so they fallback to the regular invocation logic.

In Env.get_command_from_bin(), there is some Windows-specific logic that attempts to find <name>.exe in the venv's Scripts directory and resolve its full path. This approach would work. However, the entrypoint is actually <name>.cmd, and so it dodges this particular check.

I think the most reliable approach to resolve this issue would be to use shutil.which(bin, path=env.get("PATH")) in some capacity, as mentioned in the CPython issue. This ensures the script's full path is correctly resolved based on the virtual environment's path, before the call to subprocess.Popen, rather than delegating that logic to the shell. This approach might also allow to remove (or at least trim down) the logic in Env.get_command_from_bin() and Env._bin().

Reproducing

The difficulty in reproducing this issue is getting the PATH to be this long in the first place, as it cannot feasibly be done from cmd.exe, due to its own command length limitations. To work around that, here is a script which creates a ridiculously long path and invokes poetry run ... for the scripts as defined in the pyproject.toml below.

# do_poetry_run.py
import os, subprocess, shutil

fake_path = "C:\\NotAPath"

env = os.environ.copy()
path_elems = [env["PATH"]]
for _ in range(1000):
    path_elems.append(fake_path)

env["PATH"] = os.pathsep.join(path_elems)
print("PATH length:", len(env["PATH"]))

poetry = shutil.which("poetry") or "poetry"
try:
    subprocess.run([poetry, "run", "pepscript"], env=env)
except Exception as e:
    print(f"Error running pepscript: {e}")

try:
    subprocess.run([poetry, "run", "poetryscript"], env=env)
except Exception as e:
    print(f"Error running poetryscript: {e}")
Workarounds
  • Define the script in tool.poetry.scripts rather than project.scripts
  • Don't have an absurdly long PATH :)
Poetry Installation Method

pipx

Operating System

Windows 11

Poetry Version

2.1.3

Poetry Configuration
cache-dir = "C:\\Users\\martin.boisvert\\AppData\\Local\\pypoetry\\Cache"
data-dir = "C:\\Users\\martin.boisvert\\AppData\\Roaming\\pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}\\python"  # C:\Users\martin.boisvert\AppData\Roaming\pypoetry\python
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs"  # C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false
Python Sysconfig
sysconfig.log
Platform: "win-amd64"
Python version: "3.13"
Current installation scheme: "venv"

Paths: 
    data = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
    include = "C:\Python313\Include"
    platinclude = "C:\Python313\Include"
    platlib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib\site-packages"
    platstdlib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib"
    purelib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib\site-packages"
    scripts = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Scripts"   
    stdlib = "C:\Python313\Lib"

Variables:
    BINDIR = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Scripts"    
    BINLIBDEST = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib"    
    EXE = ".exe"
    EXT_SUFFIX = ".cp313-win_amd64.pyd"
    INCLUDEPY = "C:\Python313\Include"
    LDLIBRARY = "python313.dll"
    LIBDEST = "C:\Python313\Lib"
    LIBDIR = "C:\Python313\libs"
    LIBRARY = "python313.dll"
    Py_GIL_DISABLED = "0"
    SOABI = "cp313-win_amd64"
    TZPATH = ""
    VERSION = "313"
    VPATH = "..\.."
    abi_thread = ""
    abiflags = ""
    base = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
    exec_prefix = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"       
    implementation = "Python"
    implementation_lower = "python"
    installed_base = "C:\Python313"
    installed_platbase = "C:\Python313"
    platbase = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
    platlibdir = "DLLs"
    prefix = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
    projectbase = "C:\Python313"
    py_version = "3.13.3"
    py_version_nodot = "313"
    py_version_nodot_plat = "313"
    py_version_short = "3.13"
    srcdir = "C:\Python313"
    userbase = "C:\Users\martin.boisvert\AppData\Roaming\Python"
Example pyproject.toml
[project]
name = "py-env-truncation"
version = "0.1.0"
description = ""
requires-python = ">=3.13"

[project.scripts]
pepscript = "py_env_truncation.__main__:main"

[tool.poetry.scripts]
poetryscript = "py_env_truncation.__main__:main"

[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"
Poetry Runtime Logs
poetry-runtime.log
C:\Work\Demos\py-env-truncation
(py-env-truncation-py3.13) λ do_poetry_run.py
PATH length: 14940
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\config.toml
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\auth.toml
Using virtualenv: C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\py-env-truncation-dNVbun6Y-py3.13
'pepscript' is not recognized as an internal or external command,
operable program or batch file.
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\config.toml
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\auth.toml
Using virtualenv: C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\py-env-truncation-dNVbun6Y-py3.13
Warning: 'poetryscript' is an entry point defined in pyproject.toml, but it's not installed as a script. You may get improper `sys.argv[0]`.

The support to run uninstalled scripts will be removed in a future release.

Run `poetry install` to resolve and get rid of this message.

Hello, world!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with RunCommand.handle() and RunCommand.run_script(), then inspect src/poetry/utils/env/base_env.py, especially Env.get_command_from_bin() and Env._bin(). Run do_poetry_run.py with the example pyproject.toml to reproduce the Windows failure. Done means project.scripts entry points run successfully with a near-8192-character PATH while existing tool.poetry.scripts behavior remains working.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.