python-poetry / python-poetry/poetry
Avoid downloading wheels that do not match Python requirement or when platform constraints are specified
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.3k
- Forks
- 2.5k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 30
Description
- I have searched the issues of this repo and believe that this is not a duplicate.
- I have searched the FAQ and general documentation and believe that my question is not already covered.
Feature Request
Poetry by design includes in its lockfile every wheel matching the version requirements of a package. This is done regardless of the Python version requirement, and is generally fine when ran against pypi. However this results in a big performance penalty wen ran against a repository that does not expose package hashes (pytorch once again 👀 ), which results in Poetry downloading a lot of unnecessary wheels.
Suppose I specify python = "~3.11" as Python version and add torch==specific-version as a dependency from its public repository https://download.pytorch.org/whl/torch. As the repository does not provide hashes, Poetry will (rightfully) download wheels to compute a hash on them. However, it does it regardless of the Python specification - i.e. not only cp311 wheels are downloaded, but also cp39, cp38, ... This becomes increasingly slow as new Python versions are released.
I'm wondering if Poetry would accept a PR where the inclusion of packages in the lockfile (and thus its download) is pre-filtered by filtering on the matching wheels, or if I'm missing something for which this can/should not be done. While I agree that the PyTorch team should improve its repository, I think it doesn't hurt for Poetry to stay on the safe side and exploit this performance improvement if possible (as from real world experience PyTorch is not the only repository that does not expose hashes unfortunately).
Related:
- Only search source for designated platform and python #8871
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing Poetry's lockfile package inclusion and wheel download/hash path, focusing on how Python requirements and platform constraints are currently handled. Compare the proposed filtering with the related issue #8871. Done means irrelevant wheels are excluded before download while matching wheels remain available for locking and hash computation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100