huggingface / huggingface/optimum-nvidia

Missing "third-party" Directory When Building TensorRT-LLM

Open
#178 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1k
Forks
103
Avg merge
2m
Merged PRs (30d)
1

Description

Hello,

I'm trying to build the TensorRT-LLM 0.17.0 branch (https://github.com/huggingface/optimum-nvidia/tree/trtllm-0.17.0) to make it compatible with Text Generation Inference v3.3.4 (https://github.com/huggingface/text-generation-inference/releases/tag/v3.3.4), which uses TensorRT-LLM 0.17.0.

**Steps I followed:**
I followed the build instructions from the README (after switching to the correct branch):

```bash
TARGET_SM="90-real;89-real"
git clone --recursive --depth=1 https://github.com/huggingface/optimum-nvidia.git
cd optimum-nvidia/third-party/tensorrt-llm
make -C docker release_build CUDA_ARCHS=$TARGET_SM
cd ../.. && docker build -t : -f docker/Dockerfile .
```

**Problem:**
The "third-party" directory doesn't exist in the repository structure.

**Environment:**
- Branch: trtllm-0.17.0
- Target: Compatibility with TGI v3.3.4

Could you please clarify the correct build process for this branch, or confirm if the documentation needs to be updated? Are there alternative instructions for building optimum-nvidia with TensorRT-LLM 0.17.0 support?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.