NVIDIA / NVIDIA/apex

Error installing Nvidia Apex for fairseq

Open
#1,024 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
9k
Forks
1.5k
Avg merge
2d 4h
Merged PRs (30d)
3

Description

I am trying to install the NVIDIA's apex library for faster training in fairseq. The error to the command "pip3 install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" --global-option="--deprecated_fused_adam" --global-option="--xentropy" --global-option="--fast_multihead_attn" ./ " is the following:

/usr/lib/python3/dist-packages/pip/_internal/commands/install.py:255: UserWarning: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
cmdoptions.check_install_build_global(options)
Non-user install because site-packages writeable
Created temporary directory: /tmp/pip-ephem-wheel-cache-m63t2gu6
Created temporary directory: /tmp/pip-req-tracker-38_p0cu6
Initialized build tracking at /tmp/pip-req-tracker-38_p0cu6
Created build tracker: /tmp/pip-req-tracker-38_p0cu6
Entered build tracker: /tmp/pip-req-tracker-38_p0cu6
Created temporary directory: /tmp/pip-install-34zv2fpx
Processing /mnt/c/Users/34658/Desktop/HAP-LAP/morphology/apex
Created temporary directory: /tmp/pip-req-build-e8ue0er1
Added file:///mnt/c/Users/34658/Desktop/HAP-LAP/morphology/apex to build tracker '/tmp/pip-req-tracker-38_p0cu6'
Running setup.py (path:/tmp/pip-req-build-e8ue0er1/setup.py) egg_info for package from file:///mnt/c/Users/34658/Desktop/HAP-LAP/morphology/apex
Running command python setup.py egg_info
/usr/local/lib/python3.8/dist-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-req-build-e8ue0er1/setup.py", line 35, in
_, bare_metal_major, _ = get_cuda_bare_metal_version(cpp_extension.CUDA_HOME)
File "/tmp/pip-req-build-e8ue0er1/setup.py", line 14, in get_cuda_bare_metal_version
raw_output = subprocess.check_output([cuda_dir + "/bin/nvcc", "-V"], universal_newlines=True)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Warning: Torch did not find available GPUs on this system.
 If your intention is to cross-compile, this is not an error.
By default, Apex will cross-compile for Pascal (compute capabilities 6.0, 6.1, 6.2),
Volta (compute capability 7.0), Turing (compute capability 7.5),
and, if the CUDA version is >= 11.0, Ampere (compute capability 8.0).
If you wish to cross-compile for a single specific architecture,
export TORCH_CUDA_ARCH_LIST="compute capability" before running setup.py.

Cleaning up...
Removing source in /tmp/pip-req-build-e8ue0er1
Removed file:///mnt/c/Users/34658/Desktop/HAP-LAP/morphology/apex from build tracker '/tmp/pip-req-tracker-38_p0cu6'
Removed build tracker: '/tmp/pip-req-tracker-38_p0cu6'
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Exception information:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 186, in _main
status = self.run(options, args)
File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 357, in run
resolver.resolve(requirement_set)
File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/usr/lib/python3/dist-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
abstract_dist = self.preparer.prepare_linked_requirement(req)
File "/usr/lib/python3/dist-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
abstract_dist = _get_prepared_distribution(
File "/usr/lib/python3/dist-packages/pip/_internal/operations/prepare.py", line 95, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/usr/lib/python3/dist-packages/pip/_internal/distributions/sdist.py", line 40, in prepare_distribution_metadata
self.req.prepare_metadata()
File "/usr/lib/python3/dist-packages/pip/_internal/req/req_install.py", line 564, in prepare_metadata
self.metadata_directory = self._generate_metadata()
File "/usr/lib/python3/dist-packages/pip/_internal/req/req_install.py", line 538, in _generate_metadata
return generate_metadata_legacy(
File "/usr/lib/python3/dist-packages/pip/_internal/operations/build/metadata_legacy.py", line 115, in generate_metadata
call_subprocess(
File "/usr/lib/python3/dist-packages/pip/_internal/utils/subprocess.py", line 242, in call_subprocess
raise InstallationError(exc_msg)
pip._internal.exceptions.InstallationError: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I have tried with sudo and nothing changed. The steps for installation that I followed are in this link: https://github.com/pytorch/fairseq. Thank you in advance.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with setup.py, especially get_cuda_bare_metal_version and the pip command shown in the report. Reproduce the python setup.py egg_info failure and inspect how the missing CUDA_HOME value is handled; done means the reported Apex installation command completes successfully or the failure is reported clearly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
build-system, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.