pytorch / pytorch/vision

Cannot find libQt5Core.so.5 during build

Open
#5,490 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🐛 Describe the bug

Error log:

Processing /tmp/scratch/vision
  Running command python setup.py egg_info
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/scratch/vision/setup.py", line 9, in <module>
      import torch
    File "/usr/local/lib/python3.9/dist-packages/torch/__init__.py", line 199, in <module>
      from torch._C import *  # noqa: F403
  ImportError: libQt5Core.so.5: cannot open shared object file: No such file or directory
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/bin/python3 -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize
  
  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)
  
  __file__ = %r
  sys.argv[0] = __file__
  
  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"
  
  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/tmp/scratch/vision/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /tmp/pip-pip-egg-info-snwhrsxy
  cwd: /tmp/scratch/vision/
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Based on my understanding, this is due to a special combination of environment.
We build on CentOS 7 host but inside a Debian 11 docker.
Debian11 ships Qt 5.15.2, while CentOS 7 uses 3.10 kernel.
From a certain version of Qt5, it starts to use a new kernel feature (renameat2) only available a few versions later.
So even the .so is there, it'll be ignored on 3.10 kernel.

Ubuntu 18.04 has Qt 5.9 and builds just fine, same docker can build on Debian 11 host as well.

Because docker can be built by one host and run elsewhere, it does not have to fail, that's why I'm filing this as bug.

Running uic (a simple app from Qt5) and ldconfig can demonstrate this lib-not-found behavior.

root@30f6f97c32b0:/etc/roaster/scripts# uic
/usr/lib/qt5/bin/uic: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory
root@30f6f97c32b0:/etc/roaster/scripts# ldconfig -p | grep -i libQt5Core
        libQt5Core.so.5 (libc6,x86-64, OS ABI: Linux 3.17.0) => /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
        libQt5Core.so (libc6,x86-64, OS ABI: Linux 3.17.0) => /usr/lib/x86_64-linux-gnu/libQt5Core.so

Here's some related discussion:

Versions

Current master

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with setup.py, where importing torch triggers the reported failure, and reproduce the environment using the documented CentOS 7 host and Debian 11 container combination. Run uic and ldconfig to compare the loader failure with the installed libQt5Core.so.5 metadata. Done means the vision build completes in this environment or the issue is narrowed to an explicitly documented compatibility limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, linux, python, pytorch
Domain
build-system, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.