python-poetry / python-poetry/poetry

Project's and it's depedencies' symlinks are resolved, which is unwanted

Open
#8,957 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

area/deps kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

My root project is a symlinked directory in a workspace. So that workspace might look like this

.
├── some_other_not_relevant_pkg -> /some_folder/repos/github.com/user/some_other_not_relevant_pkg/
├── other_pkg -> /some_folder/repos/gitlab.com/some_other_projects/some_repo/other_pkg
└── root_pkg -> /some_folder/repos/gitlab.com/some_projects/root_pkg/

The root_pkg has a path dependency on other_pkg via the path ../other_pkg. I run Poetry from the root_pkg folder. Poetry doesn't resolve the path dependency relative to the current PWD, but resolves the PWD to the realpath of PWD and then tries to find the other_pkg Though the path dependency is only valid relative to the path in the workspace.

So Poetry should honor the the directory structure in my workspace and not resolve all paths to their realpath. I don't know whether this is the desired behaviour, but an option to disable it would also be very helpful.

poetry lock -vvv or poetry lock -vvv -C "/absolute_path_to_workspace/root_pkg" (Same output, except from Loading configuration file... line)

Path "realpath of root_pkg"/../other_pkg for other_pkg does not exist
Loading configuration file "realpath or symlinked path of root_pkg"/poetry.toml
Adding repository PyPI (https://pypi.org/simple/) and setting it as the default one
Adding repository vbti (https://pypi.custom_url.nl) and setting it as supplemental
Using virtualenv: /home/user/my_venv
Updating dependencies
Resolving dependencies...
   1: fact: root_pkg is 0.1.0
   1: derived: root_pkg
   1: Version solving took 0.002 seconds.
   1: Tried 1 solutions.

  Stack trace:

  18  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  17  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 

  16  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 

  15  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code

  14  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  13  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/base_command.py:117 in run
       115│         io.input.validate()
       116│ 
     → 117│         return self.execute(io) or 0
       118│ 
       119│     def merge_application_definition(self, merge_args: bool = True) -> None:

  12  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:61 in execute
        59│ 
        60│         try:
     →  61│             return self.handle()
        62│         except KeyboardInterrupt:
        63│             return 1

  11  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/commands/lock.py:55 in handle
        53│         self.installer.lock(update=not self.option("no-update"))
        54│ 
     →  55│         return self.installer.run()
        56│ 

  10  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/installer.py:104 in run
       102│             self.verbose(True)
       103│ 
     → 104│         return self._do_install()
       105│ 
       106│     def dry_run(self, dry_run: bool = True) -> Installer:

   9  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/installer.py:241 in _do_install
       239│                 source_root=self._env.path.joinpath("src")
       240│             ):
     → 241│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       242│         else:
       243│             self._io.write_line("Installing dependencies from lock file")

   8  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/solver.py:72 in solve
        70│         with self._progress(), self._provider.use_latest_for(use_latest or []):
        71│             start = time.time()
     →  72│             packages, depths = self._solve()
        73│             end = time.time()
        74│ 

   7  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/solver.py:155 in _solve
       153│ 
       154│         try:
     → 155│             result = resolve_version(self._package, self._provider)
       156│ 
       157│             packages = result.packages

   6  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/mixology/__init__.py:18 in resolve_version
        16│     solver = VersionSolver(root, provider)
        17│ 
     →  18│     return solver.solve()
        19│ 

   5  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/mixology/version_solver.py:164 in solve
       162│             while next is not None:
       163│                 self._propagate(next)
     → 164│                 next = self._choose_package_version()
       165│ 
       166│             return self._result()

   4  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/mixology/version_solver.py:503 in _choose_package_version
       501│             package = locked
       502│ 
     → 503│         package = self._provider.complete_package(package)
       504│ 
       505│         conflict = False

   3  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/provider.py:571 in complete_package
       569│                     if locked is not None and locked.package.is_same_package_as(dep):
       570│                         continue
     → 571│                     self.search_for_direct_origin_dependency(dep)
       572│ 
       573│         dependencies = self._get_dependencies_with_overrides(

   2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/provider.py:254 in search_for_direct_origin_dependency
       252│         elif dependency.is_directory():
       253│             dependency = cast("DirectoryDependency", dependency)
     → 254│             package = self._search_for_directory(dependency)
       255│ 
       256│         elif dependency.is_url():

   1  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/puzzle/provider.py:360 in _search_for_directory
       358│ 
       359│     def _search_for_directory(self, dependency: DirectoryDependency) -> Package:
     → 360│         dependency.validate(raise_error=True)
       361│         package = self._direct_origin.get_package_from_directory(dependency.full_path)
       362│ 

  ValueError

  Path "realpath of root_pkg"/../other_pkg for other_pkg does not exist

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/core/packages/path_dependency.py:80 in validate
       76│     def validate(self, *, raise_error: bool) -> bool:
       77│         if not self._validation_error:
       78│             return True
       79│         if raise_error:
    →  80│             raise ValueError(self._validation_error)
       81│         logger.warning(self._validation_error)
       82│         return False
       83│ 
       84│     @property

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the workspace layout from the issue with the linked pyproject.toml and run poetry lock -vvv from the symlinked root_pkg. Read the path handling shown in poetry/core/packages/path_dependency.py; done means the ../other_pkg dependency resolves through the workspace symlink structure without reporting the realpath as missing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.