python-poetry / python-poetry/poetry

speed-up by early pruning based on markers

Open
#4,952 5 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/feature status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have searched the documentation and believe that my question is not covered.

Feature Request

As of now poetry spends a lot of time on overrides which doesn't match the environmental version or platform requirements.
They are filtered in the very end of the process, after writing possible scenarios to the lock file. It would be great to have early pruning of such overrides, particularly as more and more packages go for conditional logic in their requriements.
The issue fits the broader call for better support of markers, such as #4670

Consider this example run within a virtual environment with Python 3.8

[tool.poetry.dependencies]
python = "~3.8"
pandas = [
    {version="1.3.*",markers="python_version=='3.9.*'"},
    {version="1.2.*",markers="python_version=='3.7.*'"},
    {version="1.1.*",markers="python_version=='3.10.*'"},
]
....

Petry correctly concludes the empty set of matching packages, however it splits the graph in 6 branches (due to explicit version split and one more implicit in pandas numpy dependency) - and this escalates the more markers we have. It would help to compare the markers early and ignore some of the branches.

NOTE: this is a broader call for being able to support restricted target environments and narrow the lock to prescribed architectures, python versions and so on.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the example with Python 3.8 and trace how marker-incompatible overrides create resolver branches and lock-file scenarios. The work is done when incompatible branches are pruned before scenario generation while the empty matching result remains correct; the issue does not name specific files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.