[Question] tensorrt and tensorrtllm in the same image
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
How would you like to use TensorRT-LLM
hi,
im currently using nvcr.io/nvidia/tritonserver:25.09-trtllm-python-py3.
is there a way to use tensorrt and tensorrt-llm using the same image? i have a gemma3 and modernbert model i want to use on a single gpu using the same docker image. can i build a customer image for tritonserver - who would know that here?
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
cc @karljang @MrGeva @QiJune
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the nvcr.io/nvidia/tritonserver:25.09-trtllm-python-py3 image and review the linked TensorRT-LLM documentation and examples. Determine whether TensorRT and TensorRT-LLM can coexist for the Gemma3 and ModernBERT models on one GPU, and document whether a custom TritonServer image is supported and how it should be built.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- backend, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100