python-poetry / python-poetry/poetry

Poetry does not output values of env markers under which a version conflict occurs

Open
#5,364 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

The following bug is reproducible on both latest stable (1.1.13) and pre-release (1.2.0b1) versions of poetry.

Problem

Given a simple pyproject.toml, containing only:

[tool.poetry]
name = "meta"
version = "0.0.1"
description = "No code, just dependencies."
authors = ["Tomas Babej <tomas@test.com>"]
license = "MIT"


[tool.poetry.dependencies]
python = "3.9.12"
numpy = "1.19.5"
pandas = "1.4.1"

poetry update fails with:

  SolverProblemError

  Because pandas (1.4.1) depends on numpy (>=1.20.0)
   and meta depends on numpy (1.19.5), pandas is forbidden.

The pineed versions of numpy and pandas are, however, compatible on my platform machine (pandas requires >=1.18.5), as evidenced by installing these dependencies using pip into the same virtual environment:

pip3 install numpy==1.19.5 pandas==1.4.1
Looking in indexes: https://pypi.org/simple, https://pypi.proteinqure.com/
Collecting numpy==1.19.5
  Downloading numpy-1.19.5-cp39-cp39-manylinux2010_x86_64.whl (14.9 MB)
     |████████████████████████████████| 14.9 MB 5.0 MB/s            
Collecting pandas==1.4.1
  Using cached pandas-1.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
Requirement already satisfied: pytz>=2020.1 in /home/tbabej/.cache/pypoetry/virtualenvs/meta-WEj54WUm-py3.9/lib/python3.9/site-packages (from pandas==1.4.1) (2022.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /home/tbabej/.cache/pypoetry/virtualenvs/meta-WEj54WUm-py3.9/lib/python3.9/site-packages (from pandas==1.4.1) (2.8.2)
Requirement already satisfied: six>=1.5 in /home/tbabej/.cache/pypoetry/virtualenvs/meta-WEj54WUm-py3.9/lib/python3.9/site-packages (from python-dateutil>=2.8.1->pandas==1.4.1) (1.16.0)
Installing collected packages: numpy, pandas
  Attempting uninstall: numpy
    Found existing installation: numpy 1.19.2
    Uninstalling numpy-1.19.2:
      Successfully uninstalled numpy-1.19.2
  Attempting uninstall: pandas
    Found existing installation: pandas 1.4.0
    Uninstalling pandas-1.4.0:
      Successfully uninstalled pandas-1.4.0
Successfully installed numpy-1.19.5 pandas-1.4.1

Further investigation

Verification that python 3.9 wheel for pandas 1.4.1 allows numpy 1.19.5

Downloading the wheel for numpy 1.19.5 on Python3.9 (python3.9 -m pip download pandas==1.4.1), one can see the following dependencies in the METADATA file:

Requires-Dist: python-dateutil (>=2.8.1)
Requires-Dist: pytz (>=2020.1)
Requires-Dist: numpy (>=1.18.5) ; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10"
Requires-Dist: numpy (>=1.19.2) ; platform_machine == "aarch64" and python_version < "3.10"
Requires-Dist: numpy (>=1.20.0) ; platform_machine == "arm64" and python_version < "3.10"
Requires-Dist: numpy (>=1.21.0) ; python_version >= "3.10"

so this seems to be related to handling of markers. The platform_machine value for my workstation is:

>>> import platform
>>> platform.machine()
'x86_64'

so anything above 1.18.5 is a valid dependency on this platform.

Hints as to what is happening under the hood

Running poetry update with increased verbosity reveals the following:

$ poetry update -vvv
Using virtualenv: /home/tbabej/.cache/pypoetry/virtualenvs/meta-WEj54WUm-py3.9
Updating dependencies
Resolving dependencies...
   1: fact: meta is 0.0.1
   1: derived: meta
   1: fact: meta depends on numpy (1.19.5)
   1: fact: meta depends on pandas (1.4.1)
   1: selecting meta (0.0.1)
   1: derived: pandas (==1.4.1)
   1: derived: numpy (==1.19.5)
PyPI: 1 packages found for pandas 1.4.1
PyPI: Getting info for pandas (1.4.1) from PyPI
   0: Duplicate dependencies for numpy
   0: Different requirements found for numpy (>=1.18.5) with markers platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", numpy (>=1.19.2) with markers platform_machine == "aarch64" and python_version < "3.10" and numpy (>=1.20.0) with markers platform_machine == "arm64" and python_version < "3.10".
   1: Version solving took 0.153 seconds.
   1: Tried 1 solutions.
   0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.1'): {'numpy': <Dependency numpy (>=1.18.5)>}}).
   1: fact: meta is 0.0.1
   1: derived: meta
   1: fact: meta depends on numpy (1.19.5)
   1: fact: meta depends on pandas (1.4.1)
   1: selecting meta (0.0.1)
   1: derived: pandas (==1.4.1)
   1: derived: numpy (==1.19.5)
   1: fact: pandas (1.4.1) depends on python-dateutil (>=2.8.1)
   1: fact: pandas (1.4.1) depends on pytz (>=2020.1)
   1: fact: pandas (1.4.1) depends on numpy (>=1.18.5)
   1: selecting pandas (1.4.1)
   1: derived: pytz (>=2020.1)
   1: derived: python-dateutil (>=2.8.1)
PyPI: No release information found for numpy-0.9.6, skipping
PyPI: No release information found for numpy-0.9.8, skipping
PyPI: No release information found for numpy-1.0.3, skipping
PyPI: No release information found for numpy-1.0.4, skipping
PyPI: No release information found for numpy-1.0b1, skipping
PyPI: No release information found for numpy-1.0b4, skipping
PyPI: No release information found for numpy-1.0b5, skipping
PyPI: No release information found for numpy-1.0rc1, skipping
PyPI: No release information found for numpy-1.0rc2, skipping
PyPI: No release information found for numpy-1.0rc3, skipping
PyPI: No release information found for numpy-1.1.1, skipping
PyPI: No release information found for numpy-1.2.0, skipping
PyPI: No release information found for numpy-1.2.1, skipping
PyPI: No release information found for numpy-1.4.0, skipping
PyPI: 1 packages found for numpy 1.19.5
PyPI: Getting info for numpy (1.19.5) from PyPI
PyPI: No dependencies found, downloading archives
PyPI: Downloading sdist: numpy-1.19.5.zip
   1: selecting numpy (1.19.5)
PyPI: 6 packages found for pytz >=2020.1
PyPI: Getting info for pytz (2022.1) from PyPI
PyPI: No dependencies found, downloading archives
PyPI: Downloading wheel: pytz-2022.1-py2.py3-none-any.whl
   1: selecting pytz (2022.1)
PyPI: No release information found for python-dateutil-0.1, skipping
PyPI: No release information found for python-dateutil-0.3, skipping
PyPI: No release information found for python-dateutil-0.4, skipping
PyPI: No release information found for python-dateutil-0.5, skipping
PyPI: No release information found for python-dateutil-1.0, skipping
PyPI: No release information found for python-dateutil-1.1, skipping
PyPI: No release information found for python-dateutil-1.2, skipping
PyPI: No release information found for python-dateutil-2.0, skipping
PyPI: 2 packages found for python-dateutil >=2.8.1
PyPI: Getting info for python-dateutil (2.8.2) from PyPI
   1: fact: python-dateutil (2.8.2) depends on six (>=1.5)
   1: selecting python-dateutil (2.8.2)
   1: derived: six (>=1.5)
PyPI: 18 packages found for six >=1.5
PyPI: Getting info for six (1.16.0) from PyPI
PyPI: No dependencies found, downloading archives
PyPI: Downloading wheel: six-1.16.0-py2.py3-none-any.whl
   1: selecting six (1.16.0)
   1: Version solving took 15.523 seconds.
   1: Tried 1 solutions.
   0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.1'): {'numpy': <Dependency numpy (>=1.19.2)>}}).
   1: fact: meta is 0.0.1
   1: derived: meta
   1: fact: meta depends on numpy (1.19.5)
   1: fact: meta depends on pandas (1.4.1)
   1: selecting meta (0.0.1)
   1: derived: pandas (==1.4.1)
   1: derived: numpy (==1.19.5)
   1: fact: pandas (1.4.1) depends on python-dateutil (>=2.8.1)
   1: fact: pandas (1.4.1) depends on pytz (>=2020.1)
   1: fact: pandas (1.4.1) depends on numpy (>=1.19.2)
   1: selecting pandas (1.4.1)
   1: derived: pytz (>=2020.1)
   1: derived: python-dateutil (>=2.8.1)
   1: selecting numpy (1.19.5)
   1: selecting pytz (2022.1)
   1: fact: python-dateutil (2.8.2) depends on six (>=1.5)
   1: selecting python-dateutil (2.8.2)
   1: derived: six (>=1.5)
   1: selecting six (1.16.0)
   1: Version solving took 0.007 seconds.
   1: Tried 1 solutions.

This looks like a successful version resolution, but the resolution in fact attempts to continue further and eventually fails

...
   0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.1'): {'numpy': <Dependency numpy (>=1.20.0)>}}).
   1: fact: meta is 0.0.1
   1: derived: meta
   1: fact: meta depends on numpy (1.19.5)
   1: fact: meta depends on pandas (1.4.1)
   1: selecting meta (0.0.1)
   1: derived: pandas (==1.4.1)
   1: derived: numpy (==1.19.5)
   1: fact: pandas (1.4.1) depends on python-dateutil (>=2.8.1)
   1: fact: pandas (1.4.1) depends on pytz (>=2020.1)
   1: fact: pandas (1.4.1) depends on numpy (>=1.20.0)
   1: conflict: pandas (1.4.1) depends on numpy (>=1.20.0)
   1: ! not  numpy (>=1.20.0) is satisfied by  numpy (1.19.5)
   1: ! which is caused by "meta depends on numpy (1.19.5)"
   1: ! thus: pandas is forbidden
   1: !  pandas (1.4.1) is satisfied by  pandas (1.4.1)
   1: ! which is caused by "meta depends on pandas (1.4.1)"
   1: ! thus: version solving failed
   1: Version solving took 0.004 seconds.
   1: Tried 1 solutions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the shown pyproject.toml and poetry update -vvv, then follow the dependency-resolution trace around the pandas and numpy environment markers. Done means compatible platform-specific requirements resolve correctly and any remaining conflict identifies the relevant marker values.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.