huggingface / huggingface/optimum-nvidia
Pip Installation
- Dominant language
- Python
- Stars
- 1k
- Forks
- 103
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
Hi folks,
I recently discovered a `tensorrt_llm` pip package was published as described here: https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/installation.md#install-from-the-wheel-package
I was hoping given this, it would be possible to `pip install` optimum-nvidia within another container, but ran into some issues described below.
Package sanity check:
```
root@ced35d0-lcedt:/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation# python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorrt_llm
>>> tensorrt_llm.__version__
'0.7.1'
>>> import optimum
>>> import transformers
>>>
```
Try to run example:
```
root@ced35d0-lcedt:/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation# python3 llama.py
Traceback (most recent call last):
File "/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation/llama.py", line 22, in
from optimum.nvidia import setup_logging
ModuleNotFoundError: No module named 'optimum.nvidia'
root@ced35d0-lcedt:/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation# export PYTHONPATH=/mnt/triton/jira/optimum/optimum-nvidia/src:$PYTHONPATH
root@ced35d0-lcedt:/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation# python3 llama.py
Traceback (most recent call last):
File "/mnt/triton/jira/optimum/optimum-nvidia/examples/text-generation/llama.py", line 22, in
from optimum.nvidia import setup_logging
File "/mnt/triton/jira/optimum/optimum-nvidia/src/optimum/nvidia/__init__.py", line 28, in
from .runtime import TensorRTForCausalLM, TensorRTPreTrainedModel
File "/mnt/triton/jira/optimum/optimum-nvidia/src/optimum/nvidia/runtime.py", line 7, in
import tensorrt_llm.bindings as ctrrt
ModuleNotFoundError: No module named 'tensorrt_llm.bindings'
```
(I also tried `pip install`ing the root of this repo containing the `pyproject.toml`, but that didn't help either)
---
Is there a set of commands I can follow to successfully `pip install` optimum-nvidia using pre-built or pre-installed TRT LLM wheels? Any help would be greatly appreciated!
Contributor guide
Assessment
This issue has not been assessed yet.