NVIDIA/TensorRT-LLM
View on GitHubTensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
- Stars
- 14.7k
- Forks
- 2.8k
- Open beginner issues
- 21
- Indexed issues
- 596
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
- Dominant language
- Python
- License
- No license data
- Last GitHub push
- Sep 19, 2026
- Latest indexed
- Sep 19, 2026
- Contributing guide
- Contributing guide
- Code of conduct
- Code of conduct
- Beginner labels
- No beginner labels indexed
-
KV-Cache Management
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
NVIDIA/TensorRT-LLM#19281 ·
-
Customized kernels
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
NVIDIA/TensorRT-LLM#19240 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 62/100
NVIDIA/TensorRT-LLM#19225 ·
-
Infra
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
NVIDIA/TensorRT-LLM#19067 ·
-
Pytorch
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NVIDIA/TensorRT-LLM#19059 · 3 comments ·
-
Inference runtime
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NVIDIA/TensorRT-LLM#18816 ·
-
Disaggregated serving Pytorch
Difficulty 1/5 Under an hour Newbie friendliness 90/100
NVIDIA/TensorRT-LLM#18776 ·
-
LLM API
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
NVIDIA/TensorRT-LLM#18084 ·
-
LLM API
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
NVIDIA/TensorRT-LLM#17977 ·
-
Decoding/Sampling
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
NVIDIA/TensorRT-LLM#17916 ·
-
Pytorch
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA/TensorRT-LLM#17910 ·
-
Infra
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
NVIDIA/TensorRT-LLM#17765 ·
-
Customized kernels
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
NVIDIA/TensorRT-LLM#17764 ·
-
Testing
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
NVIDIA/TensorRT-LLM#17762 ·
-
Infra
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
NVIDIA/TensorRT-LLM#17761 ·
-
Testing
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
NVIDIA/TensorRT-LLM#17759 ·
-
Testing
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
NVIDIA/TensorRT-LLM#17757 ·
-
AutoDeploy
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA/TensorRT-LLM#17755 ·
-
AutoDeploy
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NVIDIA/TensorRT-LLM#17753 ·
-
Windows
Difficulty 1/5 Under an hour Newbie friendliness 90/100
NVIDIA/TensorRT-LLM#17743 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
NVIDIA/TensorRT-LLM#17027 ·
-
Doc
Difficulty 1/5 Under an hour Newbie friendliness 82/100
NVIDIA/TensorRT-LLM#15191 · 1 comment ·
-
Doc
Difficulty 1/5 Under an hour Newbie friendliness 75/100
NVIDIA/TensorRT-LLM#15189 · 1 comment ·
-
Infra
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
NVIDIA/TensorRT-LLM#13950 · 2 comments ·
-
Infra
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NVIDIA/TensorRT-LLM#13021 ·