python-poetry / python-poetry/poetry

Not all packages install with new-installer configured

Open
#6,865 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug status/triage
Dominant language
Python
Stars
34.3k
Forks
2.5k
Avg merge
2d 19h
Merged PRs (30d)
30

Description

  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

I have a pretty large set of dependencies to install, and using the "new-installer" I consistently get errors installing from pyproject/poetry.lock

The package on which this failure occurs changes from run to run, but the failure is consistent.

Seems like the same issue as #3336 , but in Poetry 1.2.2

Like that bug, I found the problem went away by setting
poetry config experimental.new-installer false.
I also turned off parallel installs through config and that seemed to work.

Steps:
virtualenv -p /usr/bin/python3.8 .venv
source .venv/bin/activate
poetry install --no-root -vvv

Expected: I expect all the installations to succeed

Actual:
At some point, the following error comes up, but on a different package.

Error Message:

 • Installing stack-data (0.5.1): Failed

  Stack trace:

  2  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1472 in _run
      1470│                 )
      1471│             else:
    → 1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )

  1  /usr/lib/python3.8/subprocess.py:411 in check_output
       409│         kwargs['input'] = '' if kwargs.get('universal_newlines', False) else b''
       410│
    →  411│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
       412│                **kwargs).stdout
       413│

  CalledProcessError

  Command '['/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv', '--no-deps', '/home/ezimmerman/.cache/pypoetry/artifacts/45/07/de/728d2708fad47c076b342501a5e508a952057bad469c472ddf79855656/stack_data-0.5.1-py3-none-any.whl']' returned non-zero exit status 1.

  at /usr/lib/python3.8/subprocess.py:512 in run
       508│             # We don't call process.wait() as .__exit__ does that for us.
       509│             raise
       510│         retcode = process.poll()
       511│         if check and retcode:
    →  512│             raise CalledProcessError(retcode, process.args,
       513│                                      output=stdout, stderr=stderr)
       514│     return CompletedProcess(process.args, retcode, stdout, stderr)
       515│
       516│

The following error occurred when trying to handle this error:


  Stack trace:

  3  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/pip.py:49 in pip_install
       47│
       48│     try:
    →  49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
       51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e

  2  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1435 in run_pip
      1433│         pip = self.get_pip_command()
      1434│         cmd = pip + list(args)
    → 1435│         return self._run(cmd, **kwargs)
      1436│
      1437│     def run_python_script(self, content: str, **kwargs: Any) -> int | str:

  1  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1712 in _run
      1710│     def _run(self, cmd: list[str], **kwargs: Any) -> int | str:
      1711│         kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
    → 1712│         return super()._run(cmd, **kwargs)
      1713│
      1714│     def get_temp_environ(

  EnvCommandError

  Command ['/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv', '--no-deps', '/home/ezimmerman/.cache/pypoetry/artifacts/45/07/de/728d2708fad47c076b342501a5e508a952057bad469c472ddf79855656/stack_data-0.5.1-py3-none-any.whl'] errored with the following return code 1, and output:
  Fatal Python error: init_import_size: Failed to import the site module
  Python runtime state: initialized
  Traceback (most recent call last):
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 769, in <module>
      main()
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 746, in main
      paths_in_sys = addsitepackages(paths_in_sys)
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 279, in addsitepackages
      addsitedir(sitedir, known_paths)
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 202, in addsitedir
      addpackage(sitedir, name, known_paths)
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 170, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "<frozen importlib._bootstrap>", line 553, in module_from_spec
  AttributeError: 'NoneType' object has no attribute 'loader'


  at ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1476 in _run
      1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )
      1475│         except CalledProcessError as e:
    → 1476│             raise EnvCommandError(e, input=input_)
      1477│
      1478│         return decode(output)
      1479│
      1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  Stack trace:

  5  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:261 in _execute_operation
      259│
      260│             try:
    → 261│                 result = self._do_execute_operation(operation)
      262│             except EnvCommandError as e:
      263│                 if e.e.returncode == -2:

  4  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:334 in _do_execute_operation
      332│             return 0
      333│
    → 334│         result: int = getattr(self, f"_execute_{method}")(operation)
      335│
      336│         if result != 0:

  3  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:454 in _execute_install
      452│
      453│     def _execute_install(self, operation: Install | Update) -> int:
    → 454│         status_code = self._install(operation)
      455│
      456│         self._save_url_reference(operation)

  2  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:496 in _install
      494│         )
      495│         self._write(operation, message)
    → 496│         return self.pip_install(archive, upgrade=operation.job_type == "update")
      497│
      498│     def _update(self, operation: Install | Update) -> int:

  1  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:123 in pip_install
      121│     ) -> int:
      122│         try:
    → 123│             pip_install(req, self._env, upgrade=upgrade, editable=editable)
      124│         except EnvCommandError as e:
      125│             output = decode(e.e.output)

  PoetryException

  Failed to install /home/ezimmerman/.cache/pypoetry/artifacts/45/07/de/728d2708fad47c076b342501a5e508a952057bad469c472ddf79855656/stack_data-0.5.1-py3-none-any.whl

  at ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

  • Installing zope-interface (4.7.2): Pending...
  • Installing soupsieve (2.3.2.post1)
  • Installing stack-data (0.5.1): Pending...
  • Installing stack-data (0.5.1): Installing...
  • Installing stack-data (0.5.1): Failed

  Stack trace:

  2  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1472 in _run
      1470│                 )
      1471│             else:
    → 1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )

  1  /usr/lib/python3.8/subprocess.py:411 in check_output
       409│         kwargs['input'] = '' if kwargs.get('universal_newlines', False) else b''
       410│
    →  411│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
       412│                **kwargs).stdout
       413│

  CalledProcessError

  Command '['/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv', '--no-deps', '/home/ezimmerman/.cache/pypoetry/artifacts/45/07/de/728d2708fad47c076b342501a5e508a952057bad469c472ddf79855656/stack_data-0.5.1-py3-none-any.whl']' returned non-zero exit status 1.

  at /usr/lib/python3.8/subprocess.py:512 in run
       508│             # We don't call process.wait() as .__exit__ does that for us.
       509│             raise
       510│         retcode = process.poll()
       511│         if check and retcode:
    →  512│             raise CalledProcessError(retcode, process.args,
       513│                                      output=stdout, stderr=stderr)
       514│     return CompletedProcess(process.args, retcode, stdout, stderr)
       515│
       516│

The following error occurred when trying to handle this error:


  Stack trace:

  3  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/pip.py:49 in pip_install
       47│
       48│     try:
    →  49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
       51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e

  2  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1435 in run_pip
      1433│         pip = self.get_pip_command()
      1434│         cmd = pip + list(args)
    → 1435│         return self._run(cmd, **kwargs)
      1436│
      1437│     def run_python_script(self, content: str, **kwargs: Any) -> int | str:

  1  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1712 in _run
      1710│     def _run(self, cmd: list[str], **kwargs: Any) -> int | str:
      1711│         kwargs["env"] = self.get_temp_environ(environ=kwargs.get("env"))
    → 1712│         return super()._run(cmd, **kwargs)
      1713│
      1714│     def get_temp_environ(

  EnvCommandError

  Command ['/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--prefix', '/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv', '--no-deps', '/home/ezimmerman/.cache/pypoetry/artifacts/45/07/de/728d2708fad47c076b342501a5e508a952057bad469c472ddf79855656/stack_data-0.5.1-py3-none-any.whl'] errored with the following return code 1, and output:
  Fatal Python error: init_import_size: Failed to import the site module
  Python runtime state: initialized
  Traceback (most recent call last):
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 769, in <module>
      main()
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 746, in main
      paths_in_sys = addsitepackages(paths_in_sys)
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 279, in addsitepackages
      addsitedir(sitedir, known_paths)
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 202, in addsitedir
      addpackage(sitedir, name, known_paths)
    File "/home/ezimmerman/github/pdsw-sonos-controller-player-s2/test/python/.venv/lib/python3.8/site.py", line 170, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "<frozen importlib._bootstrap>", line 553, in module_from_spec
  AttributeError: 'NoneType' object has no attribute 'loader'


  at ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1476 in _run
      1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )
      1475│         except CalledProcessError as e:
    → 1476│             raise EnvCommandError(e, input=input_)
      1477│
      1478│         return decode(output)
      1479│
      1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  Stack trace:

  5  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:261 in _execute_operation
      259│
      260│             try:
    → 261│                 result = self._do_execute_operation(operation)
      262│             except EnvCommandError as e:
      263│                 if e.e.returncode == -2:

  4  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:334 in _do_execute_operation
      332│             return 0
      333│
    → 334│         result: int = getattr(self, f"_execute_{method}")(operation)
      335│
      336│         if result != 0:

  3  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:454 in _execute_install
      452│
      453│     def _execute_install(self, operation: Install | Update) -> int:
    → 454│         status_code = self._install(operation)
      455│
      456│         self._save_url_reference(operation)

  2  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:496 in _install
      494│         )
      495│         self._write(operation, message)
    → 496│         return self.pip_install(archive, upgrade=operation.job_type == "update")
      497│
      498│     def _update(self, operation: Install | Update) -> int:

  1  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/executor.py:123 in pip_install
      121│     ) -> int:
      122│         try:
    → 123│             pip_install(req, self._env, upgrade=upgrade, editable=editable)
      124│         except EnvCommandError as e:
      125│             output = decode(e.e.output)

  PoetryException

  Failed to install /home/ezimmerman/.cache/pypoetry/artifacts/45/07/de/728d2708fad47c076b342501a5e508a952057bad469c472ddf79855656/stack_data-0.5.1-py3-none-any.whl

  at ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing poetry install --no-root -vvv with the supplied Python 3.8 environment and new installer enabled. Read the installation flow in poetry/installation/executor.py, then follow the pip and environment handling in poetry/utils/pip.py and poetry/utils/env.py. Done means all locked packages install successfully without requiring the new installer or parallel installs to be disabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.