clab / clab/dynet

CMake Error while installing dynet with GPU

Open
#1,598 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

user@user:~/Downloads$ CUDNN_ROOT=/usr/include/cudnn.h BACKEND=cuda pip install git+https://github.com/clab/dynet#egg=dynet
Collecting dynet from git+https://github.com/clab/dynet#egg=dynet
Cloning https://github.com/clab/dynet to /tmp/pip-build-L6yM2z/dynet
Collecting cython (from dynet)
Using cached https://files.pythonhosted.org/packages/7b/d2/060a4f311c3b4a83cb050882f1032dabd5c8045b489dc699cff60bcebdba/Cython-0.29.14-cp27-cp27mu-manylinux1_x86_64.whl
Collecting numpy (from dynet)
Using cached https://files.pythonhosted.org/packages/d7/b1/3367ea1f372957f97a6752ec725b87886e12af1415216feec9067e31df70/numpy-1.16.5-cp27-cp27mu-manylinux1_x86_64.whl
Installing collected packages: cython, numpy, dynet
Running setup.py install for dynet ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-L6yM2z/dynet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-pgTa3f-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
running install
running build
INFO:root:CMAKE_PATH='/usr/bin/cmake'
INFO:root:MAKE_PATH='/usr/bin/make'
INFO:root:MAKE_FLAGS='-j 4'
INFO:root:EIGEN3_INCLUDE_DIR='/tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit/eigen'
INFO:root:EIGEN3_DOWNLOAD_URL='https://bitbucket.org/eigen/eigen/get/b2e267dc99d4.zip'
INFO:root:CC_PATH='/usr/bin/gcc'
INFO:root:CXX_PATH='/usr/bin/g++'
INFO:root:SCRIPT_DIR='/tmp/pip-build-L6yM2z/dynet'
INFO:root:BUILD_DIR='/tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit'
INFO:root:INSTALL_PREFIX='/usr/lib/python2.7/dist-packages/../../..'
INFO:root:PYTHON='/usr/bin/python'
cmake version 3.10.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

INFO:root:Creating build directory /tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit
INFO:root:Fetching Eigen...
INFO:root:Unpacking Eigen...
INFO:root:Configuring...
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Optimization level: fast
-- BACKEND: cuda
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "10.1")
CUDA_LIBRARIES: /usr/local/cuda/lib64/libcudart_static.a;-lpthread;dl;/usr/lib/x86_64-linux-gnu/librt.so;/usr/local/cuda/lib64/libcurand.so
-- Found CUDNN (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- Successfully include CUDNN flags
-- Eigen dir is /tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit/eigen
-- Found Cython version 0.29.14

--- CUDA: CUBLAS: /usr/lib/x86_64-linux-gnu/libcublas.so;CUDA_cublas_device_LIBRARY-NOTFOUND RT: /usr/local/cuda/lib64/libcudart_static.a;dl;/usr/lib/x86_64-linux-gnu/librt.so;/usr/local/cuda/lib64/libcurand.so;/usr/lib/x86_64-linux-gnu/libcudnn.so
CMAKE_INSTALL_PREFIX="/usr"
PROJECT_SOURCE_DIR="/tmp/pip-build-L6yM2z/dynet"
PROJECT_BINARY_DIR="/tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit"
LIBS="/usr/local/cuda/lib64/libcudart_static.a\;dl\;/usr/lib/x86_64-linux-gnu/librt.so\;/usr/local/cuda/lib64/libcurand.so\;-lpthread"
EIGEN3_INCLUDE_DIR="/tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit/eigen"
MKL_LINK_DIRS=""
WITH_CUDA_BACKEND="1"
CUDA_RT_FILES="libcudart_static.a\;dl\;librt.so\;libcurand.so\;libcudnn.so"
CUDA_RT_DIRS="/usr/local/cuda/lib64\;/usr/lib/x86_64-linux-gnu"
CUDA_CUBLAS_FILES="libcublas.so\;CUDA_cublas_device_LIBRARY-NOTFOUND"
CUDA_CUBLAS_DIRS="/usr/lib/x86_64-linux-gnu"
MSVC=""
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_cublas_device_LIBRARY (ADVANCED)
linked by target "dynet" in directory /tmp/pip-build-L6yM2z/dynet/dynet

-- Configuring incomplete, errors occurred!
See also "/tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit/CMakeFiles/CMakeOutput.log".
See also "/tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit/CMakeFiles/CMakeError.log".
error: /usr/bin/cmake /tmp/pip-build-L6yM2z/dynet -DCMAKE_INSTALL_PREFIX='/usr/lib/python2.7/dist-packages/../../..' -DEIGEN3_INCLUDE_DIR='/tmp/pip-build-L6yM2z/dynet/build/py2.7u-64bit/eigen' -DPYTHON='/usr/bin/python' -DBACKEND='cuda' -DCUDNN_ROOT='/usr/include/cudnn.h'

----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-L6yM2z/dynet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-pgTa3f-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-L6yM2z/dynet/
user@user:~/Downloads$

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with setup.py and the CMake invocation shown in the installation log, then inspect CMakeFiles/CMakeError.log and CMakeOutput.log for detection of CUDA_cublas_device_LIBRARY. The issue is done when configuring the dynet target with BACKEND=cuda completes without a NOTFOUND library and the pip installation succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, python
Domain
build-system, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.