python-poetry / python-poetry/poetry
Update not working with two Artifactory sources
Open
Nobody has claimed this yet.
area/docs/faq
kind/enhancement
- Dominant language
- Python
- Stars
- 34.3k
- Forks
- 2.5k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 30
Description
-
I am on the latest Poetry version.
-
I have searched the issues of this repo and believe that this is not a duplicate.
-
If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption). -
OS version and name: macOS 10.15.7
-
Poetry version: 1.1.2
-
Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/austinbutler/aa1c53dc846eb9a3e913d0444c256b82
Issue
For installing packages from a private Artifactory, the following works:
[[tool.poetry.source]]
name = "artifactory"
url = "https://artifacts.example.com/artifactory/api/pypi/pypi/simple/"
default = true
But when trying to publish to that source it does not work. So I defined a second source that works for publishing:
[[tool.poetry.source]]
name = "artifactory-upload"
url = "https://artifacts.example.com/artifactory/api/pypi/pypi/"
Then I can publish with poetry publish --build --repository artifactory-upload.
But once that second source is added I get this from poetry update:
Using virtualenv: /Users/abutler/Documents/my-package/.venv
Updating dependencies
Resolving dependencies...
1: fact: my-package is 1.2.0
1: derived: my-package
1: fact: my-package depends on requests (^2.24.0)
1: fact: my-package depends on jpype1 (^1.0.2)
1: fact: my-package depends on black (^20.8b1)
1: fact: my-package depends on isort (^5.5.4)
1: fact: my-package depends on mypy (^0.782)
1: fact: my-package depends on pydocstyle (^5.1.1)
1: fact: my-package depends on pylint (^2.6.0)
1: fact: my-package depends on pytest (^6.1.1)
1: fact: my-package depends on black (^20.8b1)
1: fact: my-package depends on isort (^5.5.4)
1: fact: my-package depends on mypy (^0.782)
1: fact: my-package depends on pydocstyle (^5.1.1)
1: fact: my-package depends on pylint (^2.6.0)
1: fact: my-package depends on pytest (^6.1.1)
1: selecting my-package (1.2.0)
1: derived: pytest (^6.1.1)
1: derived: pylint (^2.6.0)
1: derived: pydocstyle (^5.1.1)
1: derived: mypy (^0.782)
1: derived: isort (^5.5.4)
1: derived: black (^20.8b1)
1: derived: jpype1 (^1.0.2)
1: derived: requests (^2.24.0)
artifactory: 1 packages found for pytest >=6.1.1,<7.0.0
1: fact: pytest (6.1.1) depends on attrs (>=17.4.0)
1: fact: pytest (6.1.1) depends on iniconfig (*)
1: fact: pytest (6.1.1) depends on packaging (*)
1: fact: pytest (6.1.1) depends on pluggy (>=0.12,<1.0)
1: fact: pytest (6.1.1) depends on py (>=1.8.2)
1: fact: pytest (6.1.1) depends on toml (*)
1: fact: pytest (6.1.1) depends on atomicwrites (>=1.0)
1: fact: pytest (6.1.1) depends on colorama (*)
1: selecting pytest (6.1.1)
1: derived: colorama (*)
1: derived: atomicwrites (>=1.0)
1: derived: toml (*)
1: derived: py (>=1.8.2)
1: derived: pluggy (>=0.12,<1.0)
1: derived: packaging (*)
1: derived: iniconfig (*)
1: derived: attrs (>=17.4.0)
artifactory: 1 packages found for pylint >=2.6.0,<3.0.0
1: fact: pylint (2.6.0) depends on astroid (>=2.4.0,<=2.5)
1: fact: pylint (2.6.0) depends on isort (>=4.2.5,<6)
1: fact: pylint (2.6.0) depends on mccabe (>=0.6,<0.7)
1: fact: pylint (2.6.0) depends on toml (>=0.7.1)
1: fact: pylint (2.6.0) depends on colorama (*)
1: selecting pylint (2.6.0)
1: derived: toml (>=0.7.1)
1: derived: mccabe (>=0.6,<0.7)
1: derived: astroid (>=2.4.0,<=2.5)
artifactory: 1 packages found for pydocstyle >=5.1.1,<6.0.0
1: fact: pydocstyle (5.1.1) depends on snowballstemmer (*)
1: selecting pydocstyle (5.1.1)
1: derived: snowballstemmer (*)
artifactory: 1 packages found for mypy >=0.782,<0.783
1: fact: mypy (0.782) depends on typed-ast (>=1.4.0,<1.5.0)
1: fact: mypy (0.782) depends on typing-extensions (>=3.7.4)
1: fact: mypy (0.782) depends on mypy-extensions (>=0.4.3,<0.5.0)
1: selecting mypy (0.782)
1: derived: mypy-extensions (>=0.4.3,<0.5.0)
1: derived: typing-extensions (>=3.7.4)
1: derived: typed-ast (>=1.4.0,<1.5.0)
artifactory: 4 packages found for isort >=5.5.4,<6.0.0
1: selecting isort (5.6.1)
artifactory: 1 packages found for black >=20.8b1,<21.0
1: fact: black (20.8b1) depends on click (>=7.1.2)
1: fact: black (20.8b1) depends on appdirs (*)
1: fact: black (20.8b1) depends on toml (>=0.10.1)
1: fact: black (20.8b1) depends on typed-ast (>=1.4.0)
1: fact: black (20.8b1) depends on regex (>=2020.1.8)
1: fact: black (20.8b1) depends on pathspec (>=0.6,<1)
1: fact: black (20.8b1) depends on typing_extensions (>=3.7.4)
1: fact: black (20.8b1) depends on mypy_extensions (>=0.4.3)
1: selecting black (20.8b1)
1: derived: pathspec (>=0.6,<1)
1: derived: regex (>=2020.1.8)
1: derived: toml (>=0.10.1)
1: derived: appdirs (*)
1: derived: click (>=7.1.2)
artifactory: 1 packages found for jpype1 >=1.0.2,<2.0.0
1: selecting jpype1 (1.0.2)
artifactory: 1 packages found for requests >=2.24.0,<3.0.0
1: fact: requests (2.24.0) depends on chardet (>=3.0.2,<4)
1: fact: requests (2.24.0) depends on idna (>=2.5,<3)
1: fact: requests (2.24.0) depends on urllib3 (>=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26)
1: fact: requests (2.24.0) depends on certifi (>=2017.4.17)
1: selecting requests (2.24.0)
1: derived: certifi (>=2017.4.17)
1: derived: urllib3 (>=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26)
1: derived: idna (>=2.5,<3)
1: derived: chardet (>=3.0.2,<4)
artifactory: 41 packages found for colorama *
1: selecting colorama (0.4.3)
artifactory: 7 packages found for atomicwrites >=1.0
1: selecting atomicwrites (1.4.0)
artifactory: 1 packages found for toml >=0.10.1
1: selecting toml (0.10.1)
artifactory: 2 packages found for py >=1.8.2
1: selecting py (1.9.0)
artifactory: 3 packages found for pluggy >=0.12,<1.0
1: selecting pluggy (0.13.1)
artifactory: 30 packages found for packaging *
1: fact: packaging (20.4) depends on pyparsing (>=2.0.2)
1: fact: packaging (20.4) depends on six (*)
1: selecting packaging (20.4)
1: derived: six (*)
1: derived: pyparsing (>=2.0.2)
artifactory: 3 packages found for iniconfig *
1: selecting iniconfig (1.0.1)
artifactory: 8 packages found for attrs >=17.4.0
1: selecting attrs (20.2.0)
artifactory: 2 packages found for mccabe >=0.6,<0.7
1: selecting mccabe (0.6.1)
artifactory: 3 packages found for astroid >=2.4.0,<=2.5
1: fact: astroid (2.4.2) depends on lazy-object-proxy (>=1.4.0,<1.5.0)
1: fact: astroid (2.4.2) depends on six (>=1.12,<2.0)
1: fact: astroid (2.4.2) depends on wrapt (>=1.11,<2.0)
1: selecting astroid (2.4.2)
1: derived: wrapt (>=1.11,<2.0)
1: derived: six (>=1.12,<2.0)
1: derived: lazy-object-proxy (>=1.4.0,<1.5.0)
artifactory: 8 packages found for snowballstemmer *
1: Version solving took 7.217 seconds.
1: Tried 1 solutions.
Stack trace:
15 .venv/lib/python3.8/site-packages/clikit/console_application.py:131 in run
129│ parsed_args = resolved_command.args
130│
→ 131│ status_code = command.handle(parsed_args, io)
132│ except KeyboardInterrupt:
133│ status_code = 1
14 .venv/lib/python3.8/site-packages/clikit/api/command/command.py:120 in handle
118│ def handle(self, args, io): # type: (Args, IO) -> int
119│ try:
→ 120│ status_code = self._do_handle(args, io)
121│ except KeyboardInterrupt:
122│ if io.is_debug():
13 .venv/lib/python3.8/site-packages/clikit/api/command/command.py:171 in _do_handle
169│ handler_method = self._config.handler_method
170│
→ 171│ return getattr(handler, handler_method)(args, io, self)
172│
173│ def __repr__(self): # type: () -> str
12 .venv/lib/python3.8/site-packages/cleo/commands/command.py:92 in wrap_handle
90│ self._command = command
91│
→ 92│ return self.handle()
93│
94│ def handle(self): # type: () -> Optional[int]
11 .venv/lib/python3.8/site-packages/poetry/console/commands/update.py:47 in handle
45│ self._installer.update(True)
46│
→ 47│ return self._installer.run()
48│
10 .venv/lib/python3.8/site-packages/poetry/installation/installer.py:103 in run
101│ local_repo = Repository()
102│
→ 103│ return self._do_install(local_repo)
104│
105│ def dry_run(self, dry_run=True): # type: (bool) -> Installer
9 .venv/lib/python3.8/site-packages/poetry/installation/installer.py:235 in _do_install
233│ )
234│
→ 235│ ops = solver.solve(use_latest=self._whitelist)
236│ else:
237│ self._io.write_line("Installing dependencies from lock file")
8 .venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:65 in solve
63│ with self._provider.progress():
64│ start = time.time()
→ 65│ packages, depths = self._solve(use_latest=use_latest)
66│ end = time.time()
67│
7 .venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:233 in _solve
231│
232│ try:
→ 233│ result = resolve_version(
234│ self._package, self._provider, locked=locked, use_latest=use_latest
235│ )
6 .venv/lib/python3.8/site-packages/poetry/mixology/__init__.py:7 in resolve_version
5│ solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
6│
→ 7│ return solver.solve()
8│
5 .venv/lib/python3.8/site-packages/poetry/mixology/version_solver.py:84 in solve
82│ while next is not None:
83│ self._propagate(next)
→ 84│ next = self._choose_package_version()
85│
86│ return self._result()
4 .venv/lib/python3.8/site-packages/poetry/mixology/version_solver.py:374 in _choose_package_version
372│ if locked is None or not dependency.constraint.allows(locked.version):
373│ try:
→ 374│ packages = self._provider.search_for(dependency)
375│ except ValueError as e:
376│ self._add_incompatibility(
3 .venv/lib/python3.8/site-packages/poetry/puzzle/provider.py:139 in search_for
137│ packages = self.search_for_url(dependency)
138│ else:
→ 139│ packages = self._pool.find_packages(dependency)
140│
141│ packages.sort(
2 .venv/lib/python3.8/site-packages/poetry/repositories/pool.py:165 in find_packages
163│ packages = []
164│ for repo in self._repositories:
→ 165│ packages += repo.find_packages(dependency)
166│
167│ return packages
1 .venv/lib/python3.8/site-packages/poetry/repositories/legacy_repository.py:261 in find_packages
259│ versions = self._cache.store("matches").get(key)
260│ else:
→ 261│ page = self._get("/{}/".format(dependency.name.replace(".", "-")))
262│ if page is None:
263│ return []
RepositoryError
500 Server Error: Internal Server Error for url: https://artifacts.example.com/artifactory/api/pypi/pypi/snowballstemmer/
at .venv/lib/python3.8/site-packages/poetry/repositories/legacy_repository.py:390 in _get
386│ if response.status_code == 404:
387│ return
388│ response.raise_for_status()
389│ except requests.HTTPError as e:
→ 390│ raise RepositoryError(e)
391│
392│ if response.status_code in (401, 403):
393│ self._log(
394│ "Authorization error accessing {url}".format(url=url), level="warn"
If I simply remove the artifactory-upload source from my pyproject.toml poetry update works fine. What's really confusing is at least according to the debug output it's trying to use the primary artifactory source, but then the url in the exception appears to be the artifactory-upload source. Maybe that's the bug, using the wrong repo URL.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the repository-resolution path shown in the trace: poetry/repositories/legacy_repository.py, poetry/repositories/pool.py, and poetry/puzzle/provider.py. Reproduce poetry update with both Artifactory sources and inspect how package lookup reaches the second source. Done means dependency updates no longer fail with the reported 500 error while publishing through the upload source remains possible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100