python-poetry / python-poetry/poetry

Dependency solver does not detect infinite loops during dependency resolution

Open
#7,508 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • Poetry version: Poetry (version 1.3.2)
  • Python version: 3.9
  • OS version and name: Ubuntu 22, inside a py39 conda environment
  • pyproject.toml:
    Just the relevant part:
python= "~3.8.0 || ~3.9.0"
torch = [
    {version = "1.9.0+cu111", markers = "python_version <= '3.6' and platform_machine == 'x86_64' and platform_system=='Linux'"},
    {version = "1.9.0+cu111", markers = "python_version > '3.6' and platform_machine == 'x86_64' and platform_system=='Linux'"},
    {version = "~1.9.0", markers = "python_version > '3.6' and platform_machine == 'aarch64' and platform_system=='Linux'"},
]
torchvision = [
    {version = ">=0.7.0,<1.0", markers = "python_version <= '3.6' and platform_machine == 'x86_64' and platform_system=='Linux'"},
    {version = "0.10", markers = "python_version > '3.6' and platform_machine == 'x86_64' and platform_system=='Linux'"},
    {version = "0.10.0+4.9.253tegra", markers = "python_version > '3.6' and platform_machine == 'aarch64' and platform_system=='Linux'"},
]
  • [ x] I am on the latest stable Poetry version, installed using a recommended method.
  • [ x] I have searched the issues of this repo and believe that this is not a duplicate.
  • [ x] I have consulted the FAQ and blog for any relevant entries or release notes.
  • [x ] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Dependency resolution using "poetry lock" is resulting in an infinite loop that is silent unless you pass -vvv and notice what is going on.

  1: derived: not torchvision (==0.10.0+4.9.253.tegra)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on numpy (*)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on torch (1.10.0+4.9.253-tegra)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on pillow (>=5.3.0)
   1: derived: not torchvision (==0.10.0+4.9.253.tegra)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on numpy (*)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on torch (1.10.0+4.9.253-tegra)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on pillow (>=5.3.0)
   1: derived: not torchvision (==0.10.0+4.9.253.tegra)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on numpy (*)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on torch (1.10.0+4.9.253-tegra)
   1: fact: torchvision (0.10.0+4.9.253.tegra) depends on pillow (>=5.3.0)

After doing "poetry lock" with just torch in pyproject.toml, follwed by "poetry add torchvision", I get:
$ poetry add torchvision
Using version ^0.14.1 for torchvision

Updating dependencies
Resolving dependencies... (0.0s)

Because no versions of torchvision match >0.14.1,<0.15.0
 and torchvision (0.14.1) depends on torch (1.13.1), torchvision (>=0.14.1,<0.15.0) requires torch (1.13.1).
So, because robovision-algorithms-solov2 depends on both torch (1.9.0+cu111) and torchvision (^0.14.1), version solving failed.

So somehow poetry picked a version of torchvision that's not compatible with the version of torch it already resolved.

The dependency resolver should be able to detect cycles like this and terminate with an error, even if it can't determine the cause of the cycle. I'm not familiar with the dependency solver algotihm, but perhaps it would be enough to detect if you derive the same thing twice and bail?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the supplied dependency setup with poetry lock -vvv and inspect the repeated resolver output for the torchvision cycle. The issue names no source files or tests; done means the resolver terminates with an error instead of looping silently, including when repeated derivations cannot be explained.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.