huggingface / huggingface/optimum-nvidia

Pip Installation

Open
#62 4 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1k
Forks
103
Avg merge
2m
Merged PRs (30d)
1

Description

Hi folks,

I recently discovered a `tensorrt_llm` pip package was published as described here: https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/installation.md#install-from-the-wheel-package

I was hoping given this, it would be possible to `pip install` optimum-nvidia within another container, but ran into some issues described below.

Package sanity check:
```
root@ced35d0-lcedt:/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation# python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorrt_llm
>>> tensorrt_llm.__version__
'0.7.1'
>>> import optimum
>>> import transformers
>>>
```
Try to run example:
```
root@ced35d0-lcedt:/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation# python3 llama.py
Traceback (most recent call last):
File "/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation/llama.py", line 22, in
from optimum.nvidia import setup_logging
ModuleNotFoundError: No module named 'optimum.nvidia'

root@ced35d0-lcedt:/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation# export PYTHONPATH=/mnt/triton/jira/optimum/optimum-nvidia/src:$PYTHONPATH

root@ced35d0-lcedt:/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation# python3 llama.py
Traceback (most recent call last):
File "/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation/llama.py", line 22, in
from optimum.nvidia import setup_logging
File "/mnt/triton/jira/optimum/optimum-nvidia/src/optimum/nvidia/__init__.py", line 28, in
from .runtime import TensorRTForCausalLM, TensorRTPreTrainedModel
File "/mnt/triton/jira/optimum/optimum-nvidia/src/optimum/nvidia/runtime.py", line 7, in
import tensorrt_llm.bindings as ctrrt
ModuleNotFoundError: No module named 'tensorrt_llm.bindings'
```

(I also tried `pip install`ing the root of this repo containing the `pyproject.toml`, but that didn't help either)

---

Is there a set of commands I can follow to successfully `pip install` optimum-nvidia using pre-built or pre-installed TRT LLM wheels? Any help would be greatly appreciated!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.