[Usage]: The multimodal feature in TensorRT-LLM is just a demo, and I see it only supports Qwen2-VL. If I want to deploy Qwen2.5-VL or Qwen3-VL, do I need to develop it myself? Will the official team not adapt it for new models going forward?
Open
Nobody has claimed this yet.
Multimodal
question
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
System Information:
- OS:
- Python version:
- CUDA version:
- GPU model(s):
- Driver version:
- TensorRT-LLM version:
Detailed output:
Paste the output of the above commands here
How would you like to use TensorRT-LLM
I want to run inference of a [specific model](put Hugging Face link here). I don't know how to integrate it with TensorRT-LLM or optimize it for my use case.
Specific questions:
- Model:
- Use case (e.g., chatbot, batch inference, real-time serving):
- Expected throughput/latency requirements:
- Multi-GPU setup needed:
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the TensorRT-LLM documentation and examples linked in the issue, then review the multimodal demo's stated model coverage. Confirm whether Qwen2.5-VL and Qwen3-VL are supported or identify the official guidance for adding them. Done means the issue has a concrete support or documentation answer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100