ImportError: No module named 'fused_layer_norm_cuda'

CUDA: 9.0, V9.0.176
gcc: 5.4.0
torch:1.1.0
python:3.5
Ubuntu : 16.04

$ git clone https://github.com/NVIDIA/apex
$ cd apex
$ pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

I use the command line to install apex, but get this issue

Cleaning up...
  Removing source in /tmp/pip-req-build-l2_6ouft
Removed build tracker '/tmp/pip-req-tracker-3uw_mz6k'
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-l2_6ouft/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" --cpp_ext --cuda_ext install --record /tmp/pip-record-6kq29sjc/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-req-build-l2_6ouft/
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/commands/install.py", line 366, in run
    use_user_site=options.use_user_site,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/req/__init__.py", line 49, in install_given_reqs
    **kwargs
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/req/req_install.py", line 791, in install
    spinner=spinner,
  File "/usr/local/lib/python3.5/dist-packages/pip/_internal/utils/misc.py", line 705, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip._internal.exceptions.InstallationError: Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-l2_6ouft/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" --cpp_ext --cuda_ext install --record /tmp/pip-record-6kq29sjc/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-req-build-l2_6ouft/
1 location(s) to search for versions of pip:
* https://pypi.org/simple/pip/
Getting page https://pypi.org/simple/pip/
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "GET /simple/pip/ HTTP/1.1" 200 11244
Analyzing links from page https://pypi.org/simple/pip/
  Found link https://files.pythonhosted.org/packages/3d/9d/1e313763bdfb6a48977b65829c6ce2a43eaae29ea2f907c8bbef024a7219/pip-0.2.tar.gz#sha256=88bb8d029e1bf4acd0e04d300104b7440086f94cc1ce1c5c3c31e3293aee1f81 (from https://pypi.org/simple/pip/), version: 0.2
  Found link https://files.pythonhosted.org/packages/18/ad/c0fe6cdfe1643a19ef027c7168572dac6283b80a384ddf21b75b921877da/pip-0.2.1.tar.gz#sha256=83522005c1266cc2de97e65072ff7554ac0f30ad369c3b02ff3a764b962048da (from https://pypi.org/simple/pip/), version: 0.2.1
  Found link https://files.pythonhosted.org/packages/17/05/f66144ef69b436d07f8eeeb28b7f77137f80de4bf60349ec6f0f9509e801/pip-0.3.tar.gz#sha256=183c72455cb7f8860ac1376f8c4f14d7f545aeab8ee7c22cd4caf79f35a2ed47 (from https://pypi.org/simple/pip/), version: 0.3
  Found link https://files.pythonhosted.org/packages/0a/bb/d087c9a1415f8726e683791c0b2943c53f2b76e69f527f2e2b2e9f9e7b5c/pip-0.3.1.tar.gz#sha256=34ce534f17065c78f980702928e988a6b6b2d8a9851aae5f1571a1feb9bb58d8 (from https://pypi.org/simple/pip/), version: 0.3.1
  Found link https://files.pythonhosted.org/packages/cf/c3/153571aaac6cf999f4bb09c019b1ff379b7b599ea833813a41c784eec995/pip-0.4.tar.gz#sha256=28fc67558874f71fddda7168f73595f1650523dce3bc5bf189713ecdfc1e456e (from https://pypi.org/simple/pip/), version: 0.4
  Found link https://files.pythonhosted.org/packages/8d/c7/f05c87812fa5d9562ecbc5f4f1fc1570444f53c81c834a7f662af406e3c1/pip-0.5.tar.gz#sha256=328d8412782f22568508a0d0c78a49c9920a82e44c8dfca49954fe525c152b2a (from https://pypi.org/simple/pip/), version: 0.5
  Found link https://files.pythonhosted.org/packages/9a/aa/f536b6d14fe03343367da2ff44eee28f340ae650cd017ca088b6be13084a/pip-0.5.1.tar.gz#sha256=e27650538c41fe1007a41abd4cfd0f905b822622cbe1f8e7e09d1215af207694 (from https://pypi.org/simple/pip/), version: 0.5.1
  Found link https://files.pythonhosted.org/packages/db/e6/fdf7be8a17b032c533d3f91e91e2c63dd81d3627cbe4113248a00c2d39d8/pip-0.6.tar.gz#sha256=4cf47db6815b2f435d1f44e1f35ff04823043f6161f7df9aec71a123b0c47f0d (from https://pypi.org/simple/pip/), version: 0.6
  Found link https://files.pythonhosted.org/packages/91/cd/105f4d3c75d0ae18e12623acc96f42168aaba408dd6e43c4505aa21f8e37/pip-0.6.1.tar.gz#sha256=efe47e84ffeb0ea4804f9858b8a94bebd07f5452f907ebed36d03aed06a9f9ec (from https://pypi.org/simple/pip/), version: 0.6.1
  Found link https://files.pythonhosted.org/packages/1c/c7/c0e1a9413c37828faf290f29

This is part logging.

Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import apex
>>> import importlib
>>> importlib.import_module("fused_layer_norm_cuda")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
ImportError: No module named 'fused_layer_norm_cuda'
>>>

I want to reinstall apex, so I uninstall it. However, I get the warning:
Skipping apex as it is not installed.

I have add cuda path in .bashrc file:

export LIBRARY_PATH=/usr/local/cuda/lib64${LIBRARY_PATH:+:${LIBRARY_PATH}}
# tell the OS where to find cuda shared objects at load time (runtime)
LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

#######################################
I also install apex by this command line:

python3 setup.py install --cuda_ext --cpp_ext

But it gives me a issue:

building 'apex_C' extension
creating build/temp.linux-x86_64-3.5
creating build/temp.linux-x86_64-3.5/csrc
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/lib/python3.5/dist-packages/torch/include -I/usr/local/lib/python3.5/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.5/dist-packages/torch/include/TH -I/usr/local/lib/python3.5/dist-packages/torch/include/THC -I/usr/include/python3.5m -c csrc/flatten_unflatten.cpp -o build/temp.linux-x86_64-3.5/csrc/flatten_unflatten.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=apex_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.5/csrc/flatten_unflatten.o -o build/lib.linux-x86_64-3.5/apex_C.cpython-35m-x86_64-linux-gnu.so
building 'amp_C' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/lib/python3.5/dist-packages/torch/include -I/usr/local/lib/python3.5/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.5/dist-packages/torch/include/TH -I/usr/local/lib/python3.5/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.5m -c csrc/amp_C_frontend.cpp -o build/temp.linux-x86_64-3.5/csrc/amp_C_frontend.o -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
/usr/local/cuda/bin/nvcc -I/usr/local/lib/python3.5/dist-packages/torch/include -I/usr/local/lib/python3.5/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.5/dist-packages/torch/include/TH -I/usr/local/lib/python3.5/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.5m -c csrc/multi_tensor_scale_kernel.cu -o build/temp.linux-x86_64-3.5/csrc/multi_tensor_scale_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -lineinfo -O3 --use_fast_math -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=amp_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
In file included from csrc/multi_tensor_scale_kernel.cu:3:0:
/usr/local/lib/python3.5/dist-packages/torch/include/ATen/cuda/CUDAContext.h:12:22: fatal error: cusparse.h: No such file or directory
compilation terminated.
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1

and I also try this suggestion:

export CUDA_HOME=/usr/local/cuda/bin/nvcc python3 setup.py install --cuda_ext --cpp_ext

If U have a good soluution, please tell me.
Thanks very much.

Start with setup.py and the build output for csrc/multi_tensor_scale_kernel.cu, then reproduce the --cuda_ext --cpp_ext installation command in the stated Python, Torch, GCC, and CUDA environment. Check the reported cusparse.h failure and the subsequent fused_layer_norm_cuda import; done means the extension builds successfully and the import works.