python-poetry / python-poetry/poetry

Multiple dependencies pointing to one git repo creates git lockfile error

Open
#8,635 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/installer area/vcs kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • Poetry version: 1.7.0
  • Python version: 3.9
  • OS version and name: Whatever is used by docker image python:3.9-slim

Disclaimer: I do not have enough time to sanitize the real code behind the error I encountered or create a reproducible example.

Issue

When two different dependencies point at different sub-directories and tags for the same git repository an error is produced by poetry install as it tries to use the same cloned repository for both tags simultaneously. The error does not occur when the maximum number of workers for poetry install is set to 1.

   FileExistsError
 
   [Errno 17] File exists: b'/poetry-build/.venv/src/<repo-name>/.git/refs/heads/master.lock'
 
   at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/dulwich/file.py:148 in __init__
       144│             self._lockfilename = self._filename + b".lock"
       145│         else:
       146│             self._lockfilename = self._filename + ".lock"
       147│         try:
     → 148│             fd = os.open(
       149│                 self._lockfilename,
       150│                 os.O_RDWR | os.O_CREAT | os.O_EXCL | getattr(os, "O_BINARY", 0),
       151│                 mask,
       152│             )

In this case there are multiple python packages in a single mono-repo that a service includes as dependencies. Versioning is handled by git tags. For any service that specifies more than one package from the mono-repo as dependencies (using git+ssh) the error occurs.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin with Poetry's git dependency handling during poetry install and the Dulwich lock path shown at dulwich/file.py:148; construct a minimal case with two packages from one tagged monorepo. Done means concurrent installation no longer raises FileExistsError for the shared repository, while the single-worker comparison remains understood.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.