pytorch / pytorch/vision

subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1

Open
#2,360 18 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

awaiting response needs reproduction topic: build
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🐛 Bug

When I tried to install the vision from source I got the following error:

#include <libavcodec/avcodec.h>
          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1510, in _run_ninja_build
    env=env)
  File "/home/ahmad/miniconda3/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 255, in <module>
    'clean': clean,
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/setuptools/__init__.py", line 161, in setup
    return distutils.core.setup(**attrs)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 173, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 159, in call_command
    self.run_command(cmdname)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run
    _build_ext.run(self)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 644, in build_extensions
    build_ext.build_extensions(self)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
    _build_ext.build_extension(self, ext)
  File "/home/ahmad/miniconda3/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 473, in unix_wrap_ninja_compile
    with_cuda=with_cuda)
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1228, in _write_ninja_file_and_compile_objects
    error_prefix='Error compiling objects for extension')
  File "/home/ahmad/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1524, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error compiling objects for extension

** Steps to reproduce **

Steps to reproduce the behavior:

  1. Installing Pytorch from source (with cuda 10.2, python 3.7)
  2. Cloning vision
  3. python setup.py install

Environment

OS: Ubuntu 20.04 LTS
GCC version: (Ubuntu 8.4.0-3ubuntu2) 8.4.0
CMake version: version 3.14.0

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GT 740M
Nvidia driver version: 440.33.01
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] torch==1.7.0a0+502ec8f
[conda] blas 1.0 mkl
[conda] cudatoolkit 10.0.130 0
[conda] magma-cuda102 2.5.2 1 pytorch
[conda] mkl 2020.1 217
[conda] mkl-include 2020.1 217
[conda] mkl-service 2.3.0 py37he904b0f_0
[conda] mkl_fft 1.1.0 py37h23d657b_0
[conda] mkl_random 1.1.1 py37h0573a6f_0
[conda] numpy 1.18.5 py37ha1c710e_0
[conda] numpy-base 1.18.5 py37hde5b4d6_0
[conda] torch 1.7.0a0+502ec8f pypi_0 pypi

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with setup.py and the torch.utils.cpp_extension build path shown in the traceback, then reproduce the source installation with the listed Python, PyTorch, CUDA, and Ubuntu versions. Trace why the vision extension cannot find libavcodec/avcodec.h; done means the documented source-install command completes without the ninja compilation error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
build-system, computer-vision
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.