python-poetry / python-poetry/poetry

optional=true not respected for local `file` dependency, but works with local `path` dependency

Open
#10,426 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

Description

Here's a really small reproducible test:

First, grab a .whl to depend on. For simplicity, I'm using six because it has no dependencies:

$ wget https://www.piwheels.org/simple/six/six-1.17.0-py2.py3-none-any.whl#sha256=cb514bebb7be30e172d571b0e5035990437dd202f3c9f6e911d9406254151b92

Now, make a pyproject.toml like this:

[tool.poetry]
name = "test-optional"
version = "0.0.1"

[tool.poetry.dependencies]
python = "^3.9"
six = {optional = true, file = "six-1.17.0-py2.py3-none-any.whl"}

[tool.poetry.extras]
myextra = ["six"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Now generate the lock file:

$ ~/.local/bin/poetry lock
Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Note there was no warning or error. Now look at the lock file and see that six is not considered optional:

$ cat poetry.lock
# This file is automatically @generated by Poetry 2.1.3 and should not be changed by hand.

[[package]]
name = "six"
version = "1.17.0"
description = "Python 2 and 3 compatibility utilities"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*"
groups = ["main"]
markers = "extra == \"myextra\""
files = [
    {file = "six-1.17.0-py2.py3-none-any.whl", hash = "sha256:cb514bebb7be30e172d571b0e5035990437dd202f3c9f6e911d9406254151b92"},
]

[package.source]
type = "file"
url = "six-1.17.0-py2.py3-none-any.whl"

[extras]
myextra = ["six"]

[metadata]
lock-version = "2.1"
python-versions = "^3.9"
content-hash = "3453e3714689a598766b5adad5524b8718b1cc55dc17b28c63922df187265330"

Note that it contains optional = false, which is an error.

Workarounds

If the pyproject.toml file uses path instead of file, everything works:

[tool.poetry]
name = "test-optional"
version = "0.0.1"

[tool.poetry.dependencies]
python = "^3.9"
six = {optional = true, path = "six-1.17.0-py2.py3-none-any.whl"}

[tool.poetry.extras]
myextra = ["six"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
$ ~/.local/bin/poetry lock
Resolving dependencies... (0.1s)

Writing lock file
$ cat poetry.lock 
# This file is automatically @generated by Poetry 2.1.3 and should not be changed by hand.

[[package]]
name = "six"
version = "1.17.0"
description = "Python 2 and 3 compatibility utilities"
optional = true
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*"
groups = ["main"]
markers = "extra == \"myextra\""
files = [
    {file = "six-1.17.0-py2.py3-none-any.whl", hash = "sha256:cb514bebb7be30e172d571b0e5035990437dd202f3c9f6e911d9406254151b92"},
]

[package.source]
type = "file"
url = "six-1.17.0-py2.py3-none-any.whl"

[extras]
myextra = ["six"]

[metadata]
lock-version = "2.1"
python-versions = "^3.9"
content-hash = "a7c28ec18952f6253307197c341fbf63ce14c757acdc3d1481b70335e1c1ff06"
Poetry Installation Method

install.python-poetry.org

Operating System

Ubuntu

Poetry Version

Poetry (version 2.1.3)

Poetry Configuration
cache-dir = "/home/chucky/.cache/pypoetry"
data-dir = "/home/chucky/.local/share/pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}/python"  # /home/chucky/.local/share/pypoetry/python
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/chucky/.cache/pypoetry/virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false
Python Sysconfig

Platform: "linux-x86_64"
Python version: "3.12"

Example pyproject.toml
[tool.poetry]
name = "test-optional"
version = "0.0.1"

[tool.poetry.dependencies]
python = "^3.9"
six = {optional = true, file = "six-1.17.0-py2.py3-none-any.whl"}

[tool.poetry.extras]
myextra = ["six"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Poetry Runtime Logs
poetry-runtime.log
$ ~/.local/bin/poetry lock -vvv
...
Resolving dependencies...
 1: fact: test-optional is 0.0.1
 1: derived: test-optional
 1: fact: test-optional depends on six (*)
 1: selecting test-optional (0.0.1)
 1: derived: six (*) @ file:///home/chucky/poetry-bug/six-1.17.0-py2.py3-none-any.whl
 1: selecting six (1.17.0 /home/chucky/poetry-bug/six-1.17.0-py2.py3-none-any.whl)
 1: Version solving took 0.003 seconds.
 1: Tried 1 solutions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the provided pyproject.toml using poetry lock, then compare the generated poetry.lock for file and path dependencies. Trace the dependency-resolution and lock-generation entry points responsible for the optional flag. Done means the file dependency records optional = true consistently with the path dependency, with regression coverage for the reported configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.