triton-inference-server

triton-inference-server/tensorrtllm_backend

The Triton TensorRT-LLM Backend

Stars: 944
Forks: 146
Open beginner issues: 0
Indexed issues: 320

Dominant language: No language data
License: Apache-2.0
Last GitHub push: Aug 17, 2026
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

320 open issues indexed

ci: add automated test coverage for llmapi (PyTorch backend) path Open

Difficulty 3/5 1-2 days Newbie friendliness 64/100

triton-inference-server/tensorrtllm_backend#854 · Opened Apr 16, 2026
Qwen2-VL SFT with extended vocab generates only vocab_size-1 (168063) tokens in TensorRT-LLM 0.17, while HF works fine Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 42/100

triton-inference-server/tensorrtllm_backend#849 · 2 reactions · Opened Mar 4, 2026
Question to optimize build Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

triton-inference-server/tensorrtllm_backend#843 · Opened Feb 12, 2026
build error Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

triton-inference-server/tensorrtllm_backend#810 · 5 comments · Opened Oct 16, 2025
Batching documentation confusing - can you update the docs of main repository please Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 38/100

triton-inference-server/tensorrtllm_backend#809 · Opened Oct 10, 2025
Is it possible to obtain scores from the TRT-LLM model? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#806 · Opened Oct 2, 2025
Whisper one-shot enc+dec path treats mel frames (3000) as “encoder length”, causing length assertion at 1500 and internal broadcast shape errors Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#804 · 1 comment · Opened Sep 29, 2025
LoRa weights not applied without warnings/errors when mismatch in type Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#791 · Opened Aug 21, 2025
Encoder-Decoder example doesn't actually use encoder? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#786 · Opened Aug 8, 2025
Quick start has outdated instructions Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

triton-inference-server/tensorrtllm_backend#770 · Opened Jun 30, 2025
Question about increased Docker image size in recent version Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#754 · 1 comment · 1 reaction · Opened May 28, 2025
lora_config shape mismatch when using converted LoRA at runtime Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#750 · Opened May 16, 2025
How can I deploy qwen2.5vl using tritonserver? Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

triton-inference-server/tensorrtllm_backend#747 · 2 comments · Opened May 8, 2025
max request capped at 228 while load testing using locust Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#740 · Opened Apr 23, 2025
no /app folder in container nvcr.io/nvidia/tritonserver:24.12-trtllm-python-py3 Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#739 · Opened Apr 22, 2025
Vision Preprocessor Not Initialized for LLaVA in Triton Workflow Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

triton-inference-server/tensorrtllm_backend#737 · Opened Apr 18, 2025
Failed to build TensorRT-LLM backend for Triton server. Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#728 · 1 comment · Opened Mar 24, 2025
tensorrtllm_backend doesn't work with remote repos Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 42/100

triton-inference-server/tensorrtllm_backend#727 · 1 comment · 1 reaction · Opened Mar 19, 2025
Failed to launch triton server with Qwen2-VL Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#724 · Opened Mar 16, 2025
`build.sh` not able to run due to python dependency issues Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#723 · Opened Mar 11, 2025
Fail to test DraftTarget model with triton server tensorrtllm backend Open

bug

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#720 · 2 comments · 1 reaction · Opened Mar 10, 2025
I want the dll version for Windows of tensorrt_llmbackend Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

triton-inference-server/tensorrtllm_backend#719 · Opened Mar 8, 2025
How to load lora weights when launching tritonserver Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#718 · 5 comments · 1 reaction · Opened Mar 7, 2025
TensorRT-LLM Backend with multiple LLMs (not to be confused with multi-model) Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#714 · 5 comments · Opened Feb 27, 2025
Missing lookAheadRuntimeConfig in Triton Server with TensorRT-LLM backend HTTP Request Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#711 · 2 comments · Opened Feb 18, 2025
Tritonserver Fails to Start with TensorRT-LLM Backend with lookahead_decoding mode - Assertion Failure in lookaheadDecodingLayer.cpp Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#710 · 1 comment · Opened Feb 18, 2025
Langgraph support Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#709 · Opened Feb 18, 2025
Failed to build TensorRT-LLM whisper Decoder Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 28/100

triton-inference-server/tensorrtllm_backend#707 · 3 comments · Opened Feb 14, 2025
Dockerfile problem Open

Difficulty 1/5 Under an hour Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#706 · 1 reaction · Opened Feb 14, 2025
numpy.ndarray' object is not callable in gpt2/1/lib/triton_decoder.py", line 160, in convert_triton_request Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#702 · Opened Feb 12, 2025
Error launching model on Triton on multigpu nodes Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#698 · Opened Feb 7, 2025
Dockerfile is using internal base images Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#694 · Opened Feb 6, 2025
Mllama ignores input image when deployed in triton Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 30/100

triton-inference-server/tensorrtllm_backend#692 · 2 comments · Opened Feb 5, 2025
Build triton_trt_llm image in debug mode Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#691 · Opened Feb 5, 2025
Performance of triton+trtllm on llava-onevision compared to vllm and sglang Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#689 · 1 comment · Opened Feb 3, 2025
Unable to build from source for tag `v0.16.0`. Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#686 · 1 comment · Opened Jan 30, 2025
DeepSeek-R1-Distill-Qwen-32B FP16 model does not work with Triton server + tensorrtllm_backend (but it works with just TensorRT-LLM) Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#685 · 2 comments · Opened Jan 30, 2025
What is the purpose of shm-region-prefix-name and what is the prefix0_ files used for? Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

triton-inference-server/tensorrtllm_backend#684 · Opened Jan 28, 2025
"error": "Unable to parse 'inputs': attempt to access non-existing object member 'inputs'" Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#683 · 1 comment · Opened Jan 28, 2025
Beam search diversity lost with in-flight batching Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 38/100

triton-inference-server/tensorrtllm_backend#682 · 1 comment · Opened Jan 24, 2025
How to deploy a quantized FP8 multimodel llm ? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#681 · 1 comment · Opened Jan 22, 2025
obj_size <= remaining_buffer_size Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#680 · 1 comment · 1 reaction · Opened Jan 20, 2025
Assertion failed: sizeof(T) <= remaining_buffer_size Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#679 · Opened Jan 14, 2025
Inference error encountered while using the draft target model. Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#678 · 1 reaction · Opened Jan 13, 2025
Why tensorrt_llm_bls backend doesn't support speculative decoding streaming or bsz > 1? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#676 · 1 comment · Opened Jan 9, 2025
import PIL on demand Open

Difficulty 2/5 1-3 hours Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#674 · Opened Jan 2, 2025
Whisper - Missing parameters for triton deployment using tensorrt_llm backend Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#672 · 1 comment · Opened Jan 2, 2025
problem: lora_weights data type Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

triton-inference-server/tensorrtllm_backend#671 · Opened Dec 25, 2024
Mllama example does not run properly for v0.15 when using the `tensorrt_llm_bls` endpoint Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/tensorrtllm_backend#669 · Opened Dec 24, 2024
Inflight Batching not working with OpenAI-Compatible Frontend Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/tensorrtllm_backend#667 · Opened Dec 22, 2024

A short digest of beginner-friendly GitHub issues.