python-poetry / python-poetry/poetry

Poetry ignores private repository when a custom section is added in the middle of pyproject.toml

Open
#4,999 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • macOS Big Sur 11.4:
  • Poetry version 1.1.12:
  • Link of a Gist with the contents of your pyproject.toml file:

Issue

So I've been playing around with poetry-dynamic-versioning and I have noticed a weird issue where if you add a custom section to the middle of the toml file e.g. [tool.poetry-dynamic-versioning], then poetry doesn't respect the private repo even if it is set as the default in the toml. However if I move this section to the top of the file then it works. For example if I have this toml

[[tool.poetry.source]]
name = "nexus"
url = "https://bla/simple"
default = true

[[tool.poetry.source]]
name = "nexus_lib"
url  = "https://bla/local/simple"
secondary = true

[tool.poetry]
name = "bla"
version = "0.0.0"
description = ""
authors = ["me"]
packages = [
    { include = "bla/*.py" },
    { include = "*.py" }
]

exclude = ["test/**/*", "*/test/**/*"]

[tool.poetry-dynamic-versioning]
enable = true
vcs = "git"
metadata = true
style = "semver"
pattern = "^(?P<base>\\d+(\\.\\d+)*)"

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^6.0"

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning"]
build-backend = "poetry.core.masonry.api"

and I try to add a library from a private repo like poetry add bla-library -vvv, poetry ignores the private repo and uses pypi public repo instead so it fails to find my library with this error:

PyPI: No packages found for bla-library *

  Stack trace:

  6  ~/.poetry/lib/poetry/_vendor/py3.9/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  5  ~/.poetry/lib/poetry/_vendor/py3.9/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  4  ~/.poetry/lib/poetry/_vendor/py3.9/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│ 
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│ 
      173│     def __repr__(self):  # type: () -> str

  3  ~/.poetry/lib/poetry/_vendor/py3.9/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│ 
    →  92│         return self.handle()
       93│ 
       94│     def handle(self):  # type: () -> Optional[int]

  2  ~/.poetry/lib/poetry/console/commands/add.py:106 in handle
      104│             return 0
      105│ 
    → 106│         requirements = self._determine_requirements(
      107│             packages,
      108│             allow_prereleases=self.option("allow-prereleases"),

  1  ~/.poetry/lib/poetry/console/commands/init.py:328 in _determine_requirements
      326│             elif "version" not in requirement:
      327│                 # determine the best version automatically
    → 328│                 name, version = self._find_best_version_for_package(
      329│                     requirement["name"],
      330│                     allow_prereleases=allow_prereleases,

  ValueError

  Could not find a matching version of package bla-library

  at ~/.poetry/lib/poetry/console/commands/init.py:367 in _find_best_version_for_package
      363│         )
      364│ 
      365│         if not package:
      366│             # TODO: find similar
    → 367│             raise ValueError(
      368│                 "Could not find a matching version of package {}".format(name)
      369│             )
      370│ 
      371│         return package.pretty_name, selector.find_recommended_require_version(package)

However if I move tool.poetry-dynamic-versioning section to the top or bottom of the file e.g.

[tool.poetry-dynamic-versioning]
enable = true
vcs = "git"
metadata = true
style = "semver"
pattern = "^(?P<base>\\d+(\\.\\d+)*)"

[[tool.poetry.source]]
name = "nexus"
url = "https://bla/simple"
default = true

[[tool.poetry.source]]
name = "nexus_lib"
url  = "https://bla/local/simple"
secondary = true

[tool.poetry]
name = "bla"
version = "0.0.0"
description = ""
authors = ["me"]
packages = [
    { include = "bla/*.py" },
    { include = "*.py" }
]

exclude = ["test/**/*", "*/test/**/*"]

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^6.0"

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning"]
build-backend = "poetry.core.masonry.api"

then poetry uses the private repo as expected. It doesn't matter what the custom section is or what it contains, the issue remains if the section is around the middle of the toml file.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the two pyproject.toml layouts and poetry add bla-library -vvv, then compare how the custom [tool.poetry-dynamic-versioning] section affects source selection. The change is done when a section placed in the middle no longer causes Poetry to ignore the default private repository, while the reported command succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.