huggingface / huggingface/optimum-nvidia
Missing "third-party" Directory When Building TensorRT-LLM
- Dominant language
- Python
- Stars
- 1k
- Forks
- 103
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
Hello,
I'm trying to build the TensorRT-LLM 0.17.0 branch (https://github.com/huggingface/optimum-nvidia/tree/trtllm-0.17.0) to make it compatible with Text Generation Inference v3.3.4 (https://github.com/huggingface/text-generation-inference/releases/tag/v3.3.4), which uses TensorRT-LLM 0.17.0.
**Steps I followed:**
I followed the build instructions from the README (after switching to the correct branch):
```bash
TARGET_SM="90-real;89-real"
git clone --recursive --depth=1 https://github.com/huggingface/optimum-nvidia.git
cd optimum-nvidia/third-party/tensorrt-llm
make -C docker release_build CUDA_ARCHS=$TARGET_SM
cd ../.. && docker build -t : -f docker/Dockerfile .
```
**Problem:**
The "third-party" directory doesn't exist in the repository structure.
**Environment:**
- Branch: trtllm-0.17.0
- Target: Compatibility with TGI v3.3.4
Could you please clarify the correct build process for this branch, or confirm if the documentation needs to be updated? Are there alternative instructions for building optimum-nvidia with TensorRT-LLM 0.17.0 support?
Contributor guide
Assessment
This issue has not been assessed yet.