Poetry fails to create lock file when specifying 'linux' platform: Package('pywin32', '306')

Open

#8,670 opened on Nov 14, 2023 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/solver kind/bug

Dominant language: Python
Stars: 34.3k
Forks: 2.5k
Avg merge: 2d 19h
Merged PRs (30d): 30

Description

Poetry version: 1.7.0
Python version: 3.10.12
OS version and name: Ubuntu 22.04
pyproject.toml: https://gist.github.com/jeertmans/8416565ee6dbbb7ebc067899315a0acb

I am on the latest stable Poetry version, installed using a recommended method.
I have searched the issues of this repo and believe that this is not a duplicate.
I have consulted the FAQ and blog for any relevant entries or release notes.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Hello, I am developing a Python package, for which I have a dependency which is only available on Linux. This dependency is only used in GitHub workflows, hence the group name (see Gist).

When restricting the platform to "linux", I guess the following error message (full traceback below) when calling poetry lock or poetry update:

Package('pywin32', '306')

caused by changing:

- open3d-cpu = {version = "^0.17.0", python = "<3.11"}
+ open3d-cpu = {version = "^0.17.0", platform = "linux", python = "<3.11"}

If I do not specify the platform, then I cannot install my package on Windows or macOS (because open3d-cpu does not exist on those platforms).

I guess this is related to #5027, but the proposed solution (using markers) does not work properly.

For information, here is the PR that initiated the build of open3d-cpu linux-only wheels: https://github.com/isl-org/Open3D/pull/5902.

Conflicting package

After searching a bit, the error seems to be caused in pair with myst-nb, but I don't see why at the first point. For information, here is a link to the pyproject.toml of myst-nb.

Stack trace:

4 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:155 in _solve
153│
154│ try:
→ 155│ result = resolve_version(self._package, self._provider)
156│
157│ packages = result.packages

3 ~/.local/lib/python3.10/site-packages/poetry/mixology/init.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│

2 ~/.local/lib/python3.10/site-packages/poetry/mixology/version_solver.py:164 in solve
162│ while next is not None:
163│ self._propagate(next)
→ 164│ next = self._choose_package_version()
165│
166│ return self._result()

1 ~/.local/lib/python3.10/site-packages/poetry/mixology/version_solver.py:503 in _choose_package_version
501│ package = locked
502│
→ 503│ package = self._provider.complete_package(package)
504│
505│ conflict = False

OverrideNeeded

({Package('pandas', '2.1.3'): {'numpy': <Dependency numpy (>=1.22.4,<2)>}}, {Package('pandas', '2.1.3'): {'numpy': <Dependency numpy (>=1.23.2,<2)>}}, {Package('pandas', '2.1.3'): {'numpy': <Dependency numpy (>=1.26.0,<2)>}})

at ~/.local/lib/python3.10/site-packages/poetry/puzzle/provider.py:660 in complete_package
656│ current_overrides.update({dependency_package: package_overrides})
657│ overrides.append(current_overrides)
658│
659│ if overrides:
→ 660│ raise OverrideNeeded(*overrides)
661│
662│ # Modifying dependencies as needed
663│ clean_dependencies = []
664│ for dep in dependencies:

The following error occurred when trying to handle this error:

Stack trace:

13 ~/.local/lib/python3.10/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a

12 ~/.local/lib/python3.10/site-packages/poetry/console/application.py:190 in _run
188│ self._load_plugins(io)
189│
→ 190│ exit_code: int = super()._run(io)
191│ return exit_code
192│

11 ~/.local/lib/python3.10/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│

10 ~/.local/lib/python3.10/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code

9 ~/.local/lib/python3.10/site-packages/cleo/application.py:457 in _run_command
455│
456│ if command_event.command_should_run():
→ 457│ exit_code = command.run(io)
458│ else:
459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

8 ~/.local/lib/python3.10/site-packages/cleo/commands/base_command.py:117 in run
115│ io.input.validate()
116│
→ 117│ return self.execute(io) or 0
118│
119│ def merge_application_definition(self, merge_args: bool = True) -> None:

7 ~/.local/lib/python3.10/site-packages/cleo/commands/command.py:61 in execute
59│
60│ try:
→ 61│ return self.handle()
62│ except KeyboardInterrupt:
63│ return 1

6 ~/.local/lib/python3.10/site-packages/poetry/console/commands/lock.py:55 in handle
53│ self.installer.lock(update=not self.option("no-update"))
54│
→ 55│ return self.installer.run()
56│

5 ~/.local/lib/python3.10/site-packages/poetry/installation/installer.py:104 in run
102│ self.verbose(True)
103│
→ 104│ return self._do_install()
105│
106│ def dry_run(self, dry_run: bool = True) -> Installer:

4 ~/.local/lib/python3.10/site-packages/poetry/installation/installer.py:241 in _do_install
239│ source_root=self._env.path.joinpath("src")
240│ ):
→ 241│ ops = solver.solve(use_latest=self._whitelist).calculate_operations()
242│ else:
243│ self._io.write_line("Installing dependencies from lock file")

3 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:72 in solve
70│ with self._progress(), self._provider.use_latest_for(use_latest or []):
71│ start = time.time()
→ 72│ packages, depths = self._solve()
73│ end = time.time()
74│

2 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:159 in _solve
157│ packages = result.packages
158│ except OverrideNeeded as e:
→ 159│ return self._solve_in_compatibility_mode(e.overrides)
160│ except SolveFailure as e:
161│ raise SolverProblemError(e)

1 ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:133 in _solve_in_compatibility_mode
131│ )
132│ self._provider.set_overrides(override)
→ 133│ _packages, _depths = self._solve()
134│ for index, package in enumerate(_packages):
135│ if package not in packages:

KeyError

Package('pywin32', '306')

at ~/.local/lib/python3.10/site-packages/poetry/puzzle/solver.py:194 in _solve
190│ # because it includes relevant extras
191│ _dep.marker = dep.marker
192│ else:
193│ final_packages.append(package)
→ 194│ depths.append(results[package])
195│
196│ # Return the packages in their original order with associated depths
197│ return final_packages, depths
198│

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with poetry lock -vvv, then read poetry/puzzle/solver.py around _solve and poetry/puzzle/provider.py around complete_package, the locations shown in the traceback. Done means the platform-constrained dependency scenario resolves successfully without the KeyError for Package('pywin32', '306'), with regression coverage for the reported case.

Written by the indexing model from the issue text.

Assessment

Tech stack: python
Domain: cli, tooling
Issue type: Bug
Difficulty: 4/5
Estimated time: 3-5 days
Activity status: Stale
Clarity: Mostly clear
Newbie friendliness: 35/100