python-poetry / python-poetry/poetry

Extra path windows dependency

Open
#9,154 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/deps area/installer area/windows kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

Description

Our project depends on some library A which has extra path dependency. We compile wheels for linux and windows and publish its to local pypi. When we try to execute poetry update on linux we have such error.

  ValueError

  non-local file URIs are not supported on this platform: file://C:\gitlab-runner\builds\eqmEVNwE\0\***\***/src/PytorchNvCodec ```


Our project does not  depends from PytorchNvCodec. But poetry download windows wheel and try to resolve this dependencies and crashes. If host machine has windows as OS then `poetry update lock` successfully. Poetry of version 1.3.1  write to log `Invalid constraint (PytorchNvCodec @ file://C:\gitlab-runner\builds\eqmEVNwE\0\***\VideoProcessingFramework/src/PytorchNvCodec/ ; extra == 'dev') found in pynvcodec-2.0.7 dependencies, skipping` and update a lock. 

`Poetry install`  works successfully 

Example of such library and deps - https://github.com/NVIDIA/VideoProcessingFramework/blob/master/setup.py#L30 .

ValueError

Package('pynvcodec', '2.0.7') is not in list

2 /etc/poetry/venv/lib64/python3.12/site-packages/poetry/inspection/info.py:197 in to_package
195│ try:
196│ # Attempt to parse the PEP-508 requirement string
→ 197│ dependency = Dependency.create_from_pep_508(req, relative_to=root_dir)
198│ except InvalidMarker:
199│ # Invalid marker, We strip the markers hoping for the best

1 /etc/poetry/venv/lib64/python3.12/site-packages/poetry/core/packages/dependency.py:463 in create_from_pep_508
461│ elif is_file_uri:
462│ # handle RFC 8089 references
→ 463│ path = url_to_path(req.url)
464│ dep = _make_file_or_dir_dep(
465│ name=name,

ValueError

non-local file URIs are not supported on this platform: file://C:\gitlab-runner\builds\eqmEVNwE\0**/src/PytorchNvCodec/


### Workarounds

Poetry version 1.3.1 or run command `poetry update` on windows.

### Poetry Installation Method

pipx

### Operating System

Centos8

### Poetry Version

1.8.2

### Poetry Configuration

```bash session
cache-dir = "/home/***/.cache/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
repositories.vlabspypi.url = "http://pypi.***/root/public/+simple"
repositories.vlabspypi_cache.url = "http://pypi.***/root/pypi/+simple"
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/***.cache/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true
Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs
poetry update -vvv
Warning: Found deprecated key 'default' or 'secondary' in pyproject.toml configuration for source ***. Please provide the key 'priority' instead. Accepted values are: 'default', 'primary', 'secondary', 'supplemental', 'explicit'.
Warning: Found deprecated priority 'default' for source '***' in pyproject.toml. You can achieve the same effect by changing the priority to 'primary' and putting the source first.
Adding repository *** (http://***/root/pypi/+simple) and setting it as the default one
Warning: Found deprecated key 'default' or 'secondary' in pyproject.toml configuration for source ***pypi. Please provide the key 'priority' instead. Accepted values are: 'default', 'primary', 'secondary', 'supplemental', 'explicit'.
Warning: Found deprecated priority 'secondary' for source '***pypi' in pyproject.toml. Consider changing the priority to one of the non-deprecated values: 'default', 'primary', 'supplemental', 'explicit'.
Adding repository *** (http://***/root/public/+simple) and setting it as secondary
Deactivating the PyPI repository
Using virtualenv: /home/***/.cache/pypoetry/virtualenvs/****-D8aJ1awn-py3.12
Updating dependencies
Resolving dependencies...
   1: fact: *** is 0.5.2
   1: derived: ***
   0: Duplicate dependencies for ***
   0: Merging requirements for ***
   1: fact: *** depends on pip (>20.0.0)
   1: fact: *** depends on pydantic (>=2,<3)
   1: fact: *** depends on configs-core (*)
   1: fact: *** depends on werkzeug (^2.3.4)
   1: fact: *** depends on *** (>=0.1.13)
   1: fact: *** depends on *** (*)
   1: fact: *** depends on *** (*)
   1: fact: *** depends on *** (*)
   1: fact: *** depends on storages (*)
   1: fact: *** depends on prometheus-client (*)
   1: fact: *** depends on multipart (0.1.0)
   1: fact: *** depends on uvloop (*)
   1: fact: *** depends on *** (0.0.4)
   1: fact: *** depends on *** (0.0.1)
   1: fact: *** depends on *** (0.0.0)
   1: fact: *** depends on pre-commit (^1.18)
   1: fact: *** depends on flake8 (^3.7.9)
   1: fact: *** depends on autoflake (*)
   1: fact: *** depends on black (*)
   1: fact: *** depends on isort (*)
   1: fact: *** depends on urllib3 (*)
   1: fact: *** depends on vlversioneer (*)
   1: fact: *** depends on beautifulsoup4 (*)
   1: fact: *** depends on sphinx-jinja (*)
   1: selecting *** (0.5.2)
   1: derived: sphinx-jinja
   1: derived: beautifulsoup4
   1: derived: vlversioneer
   1: derived: urllib3
   1: derived: isort
   1: derived: black
   1: derived: autoflake
   1: derived: flake8 (>=3.7.9,<4.0.0)
   1: derived: pre-commit (>=1.18,<2.0)
   1: derived: *** (0.0.0) @ file:///home/***/***/***/***
   1: derived: *** (0.0.1) @ file:///home/***/***/***/***
   1: derived: *** (0.0.4) @ file:///home/***/***/***/***
   1: derived: uvloop
   1: derived: multipart (0.1.0) @ file:///home/***/***/modules/multipart
   1: derived: prometheus-client
   1: derived: ***
   1: derived: ***
   1: derived: ***
   1: derived: ***
   1: derived: *** (>=0.1.13)
   1: derived: werkzeug (>=2.3.4,<3.0.0)
   1: derived: configs-core
   1: derived: pydantic (>=2,<3)
   1: derived: pip (>20.0.0)

  ValueError

  Package('pynvcodec', '2.0.7') is not in list

  at /etc/poetry/venv/lib64/python3.12/site-packages/poetry/repositories/legacy_repository.py:66 in package
       62│         Note that this will be cached so the subsequent operations
       63│         should be much faster.
       64│         """
       65│         try:
    →  66│             index = self._packages.index(Package(name, version))
       67│
       68│             return self._packages[index]
       69│         except ValueError:
       70│             package = super().package(name, version, extras)

The following error occurred when trying to handle this error:


  Stack trace:

  21  /etc/poetry/venv/lib64/python3.12/site-packages/cleo/application.py:327 in run
       325│
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  20  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│

  19  /etc/poetry/venv/lib64/python3.12/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│

  18  /etc/poetry/venv/lib64/python3.12/site-packages/cleo/application.py:473 in _run_command
       471│
       472│         if error is not None:
     → 473│             raise error
       474│
       475│         return terminate_event.exit_code

  17  /etc/poetry/venv/lib64/python3.12/site-packages/cleo/application.py:457 in _run_command
       455│
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  16  /etc/poetry/venv/lib64/python3.12/site-packages/cleo/commands/base_command.py:117 in run
       115│         io.input.validate()
       116│
     → 117│         return self.execute(io) or 0
       118│
       119│     def merge_application_definition(self, merge_args: bool = True) -> None:

  15  /etc/poetry/venv/lib64/python3.12/site-packages/cleo/commands/command.py:61 in execute
        59│
        60│         try:
     →  61│             return self.handle()
        62│         except KeyboardInterrupt:
        63│             return 1

  14  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/console/commands/update.py:56 in handle
        54│         self.installer.update(True)
        55│
     →  56│         return self.installer.run()
        57│

  13  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/installation/installer.py:104 in run
       102│             self.verbose(True)
       103│
     → 104│         return self._do_install()
       105│
       106│     def dry_run(self, dry_run: bool = True) -> Installer:

  12  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/installation/installer.py:241 in _do_install
       239│                 source_root=self._env.path.joinpath("src")
       240│             ):
     → 241│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       242│         else:
       243│             self._io.write_line("Installing dependencies from lock file")

  11  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/puzzle/solver.py:71 in solve
        69│         with self._progress(), self._provider.use_latest_for(use_latest or []):
        70│             start = time.time()
     →  71│             packages, depths = self._solve()
        72│             end = time.time()
        73│

  10  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/puzzle/solver.py:154 in _solve
       152│
       153│         try:
     → 154│             result = resolve_version(self._package, self._provider)
       155│
       156│             packages = result.packages

   9  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/mixology/__init__.py:18 in resolve_version
        16│     solver = VersionSolver(root, provider)
        17│
     →  18│     return solver.solve()
        19│

   8  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/mixology/version_solver.py:175 in solve
       173│             while next is not None:
       174│                 self._propagate(next)
     → 175│                 next = self._choose_package_version()
       176│
       177│             return self._result()

   7  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
       512│             package = locked
       513│
     → 514│         package = self._provider.complete_package(package)
       515│
       516│         conflict = False

   6  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/puzzle/provider.py:489 in complete_package
       487│                 dependency_package = DependencyPackage(
       488│                     dependency,
     → 489│                     self._pool.package(
       490│                         package.pretty_name,
       491│                         package.version,

   5  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/repositories/repository_pool.py:204 in package
       202│         for repo in self.repositories:
       203│             try:
     → 204│                 return repo.package(name, version, extras=extras)
       205│             except PackageNotFound:
       206│                 continue

   4  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/repositories/legacy_repository.py:70 in package
        68│             return self._packages[index]
        69│         except ValueError:
     →  70│             package = super().package(name, version, extras)
        71│             package._source_type = "legacy"
        72│             package._source_url = self._url

   3  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/repositories/cached_repository.py:75 in package
        73│         extras: list[str] | None = None,
        74│     ) -> Package:
     →  75│         return self.get_release_info(canonicalize_name(name), version).to_package(
        76│             name=name, extras=extras
        77│         )

   2  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/inspection/info.py:197 in to_package
       195│             try:
       196│                 # Attempt to parse the PEP-508 requirement string
     → 197│                 dependency = Dependency.create_from_pep_508(req, relative_to=root_dir)
       198│             except InvalidMarker:
       199│                 # Invalid marker, We strip the markers hoping for the best

   1  /etc/poetry/venv/lib64/python3.12/site-packages/poetry/core/packages/dependency.py:463 in create_from_pep_508
       461│             elif is_file_uri:
       462│                 # handle RFC 8089 references
     → 463│                 path = url_to_path(req.url)
       464│                 dep = _make_file_or_dir_dep(
       465│                     name=name,

  ValueError

  non-local file URIs are not supported on this platform: file://C:\gitlab-runner\builds\eqmEVNwE\0\***\***/src/PytorchNvCodec/

  at /etc/poetry/venv/lib64/python3.12/site-packages/poetry/core/packages/utils/utils.py:82 in url_to_path
       78│     elif netloc not in {".", ".."} and sys.platform == "win32":
       79│         # If we have a UNC path, prepend UNC share notation.
       80│         netloc = "\\\\" + netloc
       81│     else:
    →  82│         raise ValueError(
       83│             f"non-local file URIs are not supported on this platform: {url}"
       84│         )
       85│
       86│     return Path(url2pathname(netloc + unquote(path)))

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with poetry/core/packages/dependency.py at create_from_pep_508 and follow the resolver path through poetry/puzzle/provider.py and poetry/repositories/legacy_repository.py, as shown in the trace. Reproduce poetry update on Linux with the Windows file dependency, then verify resolution no longer crashes while handling the platform-specific dependency correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.