python-poetry / python-poetry/poetry

Package dependencies not installed when private PyPi repository present

Open
#4,000 17 comments 10 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/needs-reproduction
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: MacOS Big Sur 11.2.3
  • Poetry version: 1.1.6

Issue

I encountered an issue with dependencies when using a private repository. I have a pyproject.toml like

[tool.poetry]
name = "my-awesome-package"
version = "0.1.0"
description = "My package."
authors = ["Me"]
packages = [
    { include = "sub_package_1" },
    { include = "sub_package_2" }
]
include = ["path_to_files/**/*.gz"]

[[tool.poetry.source]]
name = 'private_pypi'
url = 'https://mypypi.dom/pypi/simple'

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]

[build-system]
build-backend = "poetry.masonry.api"
requires = ["poetry>=1.1.2"]

And I'd like to add a new package, explicitly poetry add rasterio.
When the private repository (which has nothing to do with the desired package) is present in pyproject.toml, only
the package itself gets installed without its dependencies

poetry add rasterio                                                                                                                                                                                                                                                
Using version ^1.2.3 for rasterio

Updating dependencies
Resolving dependencies... (10.2s)

Writing lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing rasterio (1.2.3)

When I remove the private repository part

[[tool.poetry.source]]
name = 'private_pypi'
url = 'https://mypypi.dom/pypi/simple'

and try to add the package, all of its dependencies get installed correctly

poetry add rasterio
Using version ^1.2.3 for rasterio

Updating dependencies
Resolving dependencies... (3.5s)

Writing lock file

Package operations: 6 installs, 0 updates, 0 removals

  • Installing click (7.1.2)
  • Installing affine (2.3.0)
  • Installing click-plugins (1.1.1)
  • Installing cligj (0.7.1)
  • Installing snuggs (1.4.7)
  • Installing rasterio (1.2.3)

It seems that the private repository is somehow interfering with the resolution of the dependencies.
I have tried to recreate both the poetry.lock file and the virtual environment from scratch with no change in this behaviour.

Initially, I have discovered this issue through a transitive dependency (a dependency on the private pypi repository, which I was trying to install, depends on rasterio) when I tried to poetry update, the dependencies affine, click-plugins , snuggs and cligj got uninstalled, so the issue seemed to be related to #3224 and #3684, but the issue can be reproduced with this minimalistic setup.

I have specified the missing dependencies explicitly in the pyproject.toml file for the time being. This however is a suboptimal solution as it bloats pyproject.toml.

Thank you for your help!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue from the minimal pyproject.toml in the report by running poetry add rasterio with and without the private PyPI source. Compare the resolver output and lock changes; done means rasterio and its transitive dependencies are resolved and installed correctly while the private source remains configured.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.