pytorch / pytorch/vision

"libtorch.so: error adding symbols: File in wrong format" when trying to compile torchvision on Linux on IBM z machine

Open
#4,476 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic: build
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

Hello Guys,

I'm getting below error while trying to compile torchvision on linux on IBM z14 machine.
Could anyone help have a look on this?

My env:

redhat 8.4 on IBM LinuxONE
gcc (GCC) 8.4.1 20200928 (Red Hat 8.4.1-1)

command sequence to compile:

wget https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip
unzip libtorch-shared-with-deps-latest.zip

git clone git://github.com/pytorch/vision.git
cd vision
mkdir build
cd build

Add -DWITH_CUDA=on support for the CUDA if needed

cmake -DCMAKE_PREFIX_PATH=$PWD/../../libtorch ..
make
make install

cmake and make logs:


(base) [root@qiskit build]# cmake -DCMAKE_PREFIX_PATH=$PWD/../../libtorch .. 
-- Found PNG: /usr/lib64/libpng.so (found version "1.6.34") 
-- Found JPEG: /usr/lib64/libjpeg.so (found version "62") 
-- Configuring done
CMake Warning at CMakeLists.txt:73 (add_library):
  Cannot generate a safe runtime search path for target torchvision because
  files in some directories may conflict with libraries in implicit
  directories:

    runtime library [libpng16.so.16] in /usr/lib64 may be hidden by files in:
      /opt/anaconda/lib

  Some of these libraries may not be found correctly.


-- Generating done
-- Build files have been written to: /root/vision/build
(base) [root@qiskit build]# 
(base) [root@qiskit build]# 
(base) [root@qiskit build]# 
(base) [root@qiskit build]# 
(base) [root@qiskit build]# 
(base) [root@qiskit build]# 
(base) [root@qiskit build]# make 
Scanning dependencies of target torchvision
[  2%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/cpu/common_jpeg.cpp.o
[  5%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/cpu/decode_image.cpp.o
[  7%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/cpu/decode_jpeg.cpp.o
[ 10%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/cpu/decode_png.cpp.o
[ 12%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/cpu/encode_jpeg.cpp.o
[ 15%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/cpu/encode_png.cpp.o
[ 17%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/cpu/read_write_file.cpp.o
[ 20%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/cuda/decode_jpeg_cuda.cpp.o
[ 22%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/io/image/image.cpp.o
[ 25%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/alexnet.cpp.o
[ 27%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/densenet.cpp.o
[ 30%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/googlenet.cpp.o
[ 32%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/inception.cpp.o
[ 35%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/mnasnet.cpp.o
[ 37%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/mobilenet.cpp.o
[ 40%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/resnet.cpp.o
[ 42%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/shufflenetv2.cpp.o
[ 45%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/squeezenet.cpp.o
[ 47%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/models/vgg.cpp.o
[ 50%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/autograd/deform_conv2d_kernel.cpp.o
[ 52%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/autograd/ps_roi_align_kernel.cpp.o
[ 55%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/autograd/ps_roi_pool_kernel.cpp.o
[ 57%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/autograd/roi_align_kernel.cpp.o
[ 60%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/autograd/roi_pool_kernel.cpp.o
[ 62%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/cpu/deform_conv2d_kernel.cpp.o
[ 65%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/cpu/interpolate_aa_kernels.cpp.o
[ 67%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/cpu/nms_kernel.cpp.o
[ 70%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/cpu/ps_roi_align_kernel.cpp.o
[ 72%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/cpu/ps_roi_pool_kernel.cpp.o
[ 75%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/cpu/roi_align_kernel.cpp.o
[ 77%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/cpu/roi_pool_kernel.cpp.o
[ 80%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/deform_conv2d.cpp.o
[ 82%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/interpolate_aa.cpp.o
[ 85%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/nms.cpp.o
[ 87%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/ps_roi_align.cpp.o
[ 90%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/ps_roi_pool.cpp.o
[ 92%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/roi_align.cpp.o
[ 95%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/ops/roi_pool.cpp.o
[ 97%] Building CXX object CMakeFiles/torchvision.dir/torchvision/csrc/vision.cpp.o
In file included from /root/vision/torchvision/csrc/vision.cpp:1:
/root/vision/torchvision/csrc/vision.h:10:40: warning: ‘_register_ops’ initialized and declared ‘extern’
 extern "C" VISION_INLINE_VARIABLE auto _register_ops = &cuda_version;
                                        ^~~~~~~~~~~~~
[100%] Linking CXX shared library libtorchvision.so
/root/libtorch/lib/libtorch.so: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/torchvision.dir/build.make:680: libtorchvision.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:95: CMakeFiles/torchvision.dir/all] Error 2
make: *** [Makefile:149: all] Error 2
(base) [root@qiskit build]# 

Thanks
Regards

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the CMake invocation and CMakeLists.txt:73, then inspect the libtorch archive used from /root/libtorch and the link step for torchvision/csrc/vision.cpp. Reproduce the failure with the listed LinuxONE environment and determine whether the issue is an incompatible library or a project build problem; done requires a confirmed cause and a documented fix or limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, linux
Domain
build-system, computer-vision
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.