python-poetry / python-poetry/poetry

Installing package when depending on parent dir fails when it contains dep to private repo

Open
#1,209 3 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

  • #1217 by @pdavies — closed without merging
kind/bug
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I am on the latest Poetry version:
    Tried with both 0.12.16 and 1.0.0a4.
  • I have searched the issues of this repo and believe that this is not a duplicate.
    Looks similar to #1026, but not exactly the same.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

Issue

I want to add a subdirectory to an existing (private) project (f.e. private-package-from-parent-dir), define a new package there (f.e. subprojectname) and depend on the main private project using path = .. to direct poetry where to find it.

Unfortunately, poetry seems to be confused as to where to find the different dependencies.

The parent project (private-package-from-parent-dir) depends on both packages from pypi.org and a private one (private-package-published-on-privaterepo). The private one has dependencies only to public pypi packages.

Creating the lock in the subproject succeeds, but installing the dependencies fails:

> cd subprojectname
> poetry install -vvv
[...]
[EnvCommandError]
Command ['/Users/nbigaouette/private-package-from-parent-dir.git/subprojectname/.venv/bin/python', '-m', 'pip', 'install', '--no-deps', '-U', '-e', '/Users/nbigaouette/private-package-from-parent-dir.git'] errored with
the following output:
Obtaining file:///Users/nbigaouette/private-package-from-parent-dir.git
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'error'
  ERROR: Complete output from command /Users/nbigaouette/private-package-from-parent-dir.git/subprojectname/.venv/bin/python /Users/nbigaouette/private-package-from-parent-dir.git/subprojectname/.venv/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/3z/mjcq38mx3610kl4gnt9zw3gw0000gn/T/pip-build-env-zsn9ntu0/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'private-package-published-on-privaterepo>=0.4.0,<0.5.0' 'structlog>=19.1,<20.0':
  ERROR: Collecting private-package-published-on-privaterepo<0.5.0,>=0.4.0
    ERROR: Could not find a version that satisfies the requirement private-package-published-on-privaterepo<0.5.0,>=0.4.0 (from versions: none)
  ERROR: No matching distribution found for private-package-published-on-privaterepo<0.5.0,>=0.4.0
  ----------------------------------------
ERROR: Command "/Users/nbigaouette/private-package-from-parent-dir.git/subprojectname/.venv/bin/python /Users/nbigaouette/private-package-from-parent-dir.git/subprojectname/.venv/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/3z/mjcq38mx3610kl4gnt9zw3gw0000gn/T/pip-build-env-zsn9ntu0/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'private-package-published-on-privaterepo>=0.4.0,<0.5.0' 'structlog>=19.1,<20.0'" failed with error code 1 in None

As shown above, poetry is trying to install both private-package-published-on-privaterepo and structlog by using pypi.org, which are dependencies of the main project private-package-from-parent-dir. Surprisingly, both of these packages are already installed:

> poetry run python -c 'import structlog; print(structlog.__version__)'
19.1.0
> poetry run python -c 'import private_package_published_on_privaterepo; print(private_package_published_on_privaterepo.__version__)'
0.1.0

I did try to "incrementally" install the dependencies, for example by manually poetry add the structlog and private-package-published-on-privaterepo packages (which should be indirect dependencies) as to let poetry simply install the package from the parent directory but whatever I do it will always want to install the parent package's dependencies. See the gist for both parent project's and subproject's pyproject.toml files.

Poetry should have simply installed the private-package-from-parent-dir package from the parent directory (since I've used private-package-from-parent-dir = {path = ".."}), pull private-package-published-on-privaterepo from my private repo and pull structlog from pypi.org.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the parent and subproject pyproject.toml files from the linked gist, then reproduce the failure with poetry install -vvv from the subproject. Trace how the parent path dependency and its private repository dependency are resolved; done means installation uses the configured private source for the private package and PyPI for public dependencies.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.