python-poetry / python-poetry/poetry

Multiple constraints on same package cause O(exp(N)) checks

Open
#5,121 16 comments 16 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

poetry init -n
poetry add -vvv numpy ipython jupyter opencv-python notebook pandas django~3.2

Runs very long dependency resolution.

Problem is that it does a lot of override attempts (why?).
0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}).

including weird ones like
0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}).

52 override checks for this case, so, if you now have several additional slow loading packages, e.g. each override checking would take 20 seconds, you'll get 20 * 52 = 520 seconds to install.

0: Complete version solving took 13.010 seconds with 52 overrides
0: Resolved with overrides: ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}})

In more complicated cases it will be:

0: Duplicate dependencies for numpy 0: Different requirements found for numpy (>=1.18.5) with markers platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", numpy (>=1.19.2) with markers platform_machine == "aarch64" and python_version < "3.10" and numpy (>=1.20.0) with markers platform_machine == "arm64" and python_version < "3.10".
and
0: Complete version solving took 115.009 seconds with 16 overrides 0: Resolved with overrides: ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}})

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the provided poetry init and poetry add -vvv commands, then inspect the resolver output around duplicate dependencies and overrides. Done means reducing redundant override checks in this case without breaking dependency resolution correctness; the payload does not identify implementation files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.