NVIDIA-Merlin / NVIDIA-Merlin/Transformers4Rec

[DOC] Missing detail in pip install instructions

Open
#560 3 comments 0 reactions 1 assignee View on GitHub

@gabrielspmoreira is already working on this.

Since Dec 13, 2022.

status/needs-triage
Dominant language
Python
Stars
1.3k
Forks
165
Avg merge
1m
Merged PRs (30d)
2

Description

Report incorrect documentation

Location of incorrect documentation

README.md

Describe the problems or issues found in the documentation

Not incorrect, but missing from pip install instructions were the install of nvidia-pyindex and nvidia-cudnn, followed by adding cublas/lib to my $LD_LIBRARY_PATH

Steps taken to verify documentation is incorrect

This was the case on a fresh Ubuntu 20.04 machine, running Python 3.8 in an Conda env.

Suggested fix for documentation

I can raise a PR with what I'd expect (CONTRIBUTING.md says we should discuss it first), basically a prerequisites section for the pip install mentioning nvidia-pyindex and cudnn, then a note about how to update LD_LIBRARY_PATH for the error:

/nvidia/cublas/lib/libcublas.so.11: undefined symbol: cublasLtGetStatusString, version libcublasLt.so.11

So the suggested change is the "Installing with pip" section becomes:

Installing with pip

It is required to install nvidia-pyindex and nvidia-cudnn before use:

pip install nvidia-pyindex nvidia-cudnn

You can then install Transformers4Rec with the functionality to use the GPU-accelerated NVTabular dataloader.
Installation with the dataloader is highly recommended for better performance.
Those components can be installed as optional args for the pip install package. Note that installation NVTabular with pip supports only CPU version of NVTabular for now.

  • PyTorch
    pip install transformers4rec[pytorch,nvtabular]

In the event upon running Transformers4Rec you see the error:

/nvidia/cublas/lib/libcublas.so.11: undefined symbol: cublasLtGetStatusString, version libcublasLt.so.11

You may need to update the LD_LIBRARY_PATH with the path to site-packages/nvidia/cublas/lib.
pip show nvidia-cudnn will give you the location of site-packages under "Location".
Add this full path to the LD_LIBRARY_PATH environment variable as follows:

export LD_LIBRARY_PATH=path-to-site-packages/nvidia/cublas/lib/:$LD_LIBRARY_PATH

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.