pytorch / pytorch/audio

torchaudio v0.9.0 installation error (protobuf not found)

Open
#2,194 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.9k
Forks
799
Avg merge
58m
Merged PRs (30d)
3

Description

Hi, after installing PyTorch 1.9.0, I tried to install torchaudio v0.9.0 (release/v0.9) from source on a ppc64le machine. However, I received an error that CMake is unable to locate protobuf. The server shows a CMake version of 3.11.4 and a ninja version of 1.10.1, and switching CMake to a higher version of 3.18.4 shows the same error.

CMake Warning at /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Caffe2/public/protobuf.cmake:88 (message):
  Protobuf cannot be found.  Depending on whether you are building Caffe2 or
  a Caffe2 dependent library, the next warning / error will give you more
  info.
Call Stack (most recent call first):
  /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:56 (include)
  /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:67 (find_package)


CMake Error at /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:58 (message):
  Your installed Caffe2 version uses protobuf but the protobuf library cannot
  be found.  Did you accidentally remove it, or have you set the right
  CMAKE_PREFIX_PATH? If you do not have protobuf, you will need to install
  protobuf and set the library path accordingly.
Call Stack (most recent call first):
  /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:67 (find_package)

but pip install protobuf gives

Requirement already satisfied: protobuf in /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages (3.14.0)
Requirement already satisfied: six>=1.9 in /nobackup/users/junruin2/anaconda3/envs/exp_spring2022/lib/python3.7/site-packages (from protobuf) (1.15.0)

I saw the same error in (https://github.com/pytorch/audio/issues/1796) but as they installed another version of torchaudio and pytorch in the end, the issue above possibly remained for higher versions on the same system.

Here is the environment info pulled with https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py

PyTorch version: 1.9.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Red Hat Enterprise Linux release 8.3 (Ootpa) (ppc64le)
GCC version: (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)
Clang version: Could not collect
CMake version: version 3.11.4
Libc version: glibc-2.17

Python version: 3.7.11 (default, Jul 27 2021, 15:02:35)  [GCC 7.3.0] (64-bit runtime)
Python platform: Linux-4.18.0-240.el8.ppc64le-ppc64le-with-redhat-8.3-Ootpa
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] torch==1.9.0
[pip3] torchtext==0.10.0a0+4da1de3
[pip3] torchvision==0.10.0
[conda] _pytorch_select           2.0                      cuda_2    https://opence.mit.edu
[conda] cudatoolkit               10.2.89            684.g752c550    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda
[conda] numpy                     1.19.2           py37h6163131_0  
[conda] numpy-base                1.19.2           py37h75fe3a5_0  
[conda] pytorch                   1.9.0           cuda10.2_py37_1    https://opence.mit.edu
[conda] pytorch-base              1.9.0           h1234567_cuda10.2_py37_pb3.14_1    https://opence.mit.edu
[conda] torchtext                 0.10.0                   py37_1    https://opence.mit.edu
[conda] torchvision-base          0.10.0          cuda10.2_py37_1    https://opence.mit.edu

TIA for the help!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The failure is reported at CMakeLists.txt:67 and in Torch/Caffe2's protobuf.cmake; start by reproducing the torchaudio v0.9 source install in the stated ppc64le environment and tracing protobuf discovery. Done means the build can locate the required protobuf dependency, or the supported dependency and configuration requirements are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, python
Domain
build-system, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.