triton-inference-server/server

View on GitHub

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Stars: 11k
Forks: 1.8k
Open beginner issues: 3
Indexed issues: 780
Avg merge: 3d 16h
Merged PRs (30d): 28

Dominant language: Python
License: BSD-3-Clause
Last GitHub push: Sep 17, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted good first issue

Beginner friendly Everything

Open All

779 open issues indexed

k8s-onprem autoscaling metric avg_time_queue_us under-reports queue time by roughly the number of loaded models Open

Difficulty 3/5 1-2 days Newbie friendliness 70/100

triton-inference-server/server#8979 · Opened Sep 18, 2026
python_backend: error-string shm freed before parent reads it in is_ready() readiness check -> allocator corruption -> permanent hang Open

Difficulty 2/5 1-3 hours Newbie friendliness 82/100

triton-inference-server/server#8978 · 1 comment · Opened Sep 17, 2026
TensorRT backend leaks GPU resources after model initialization failure; unload does not recover memory Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

triton-inference-server/server#8969 · Opened Sep 11, 2026
Random latency regression on L40S nodes after new pod creation — node becomes persistently "bad" Open

performance

triton-inference-server/server#8947 · 10 comments · 1 assignee · Opened Aug 31, 2026
Triton TRT-LLM Multimodal guide outdated Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

triton-inference-server/server#8945 · Opened Aug 31, 2026
vLLM backend passes deprecated raw prompts to InputProcessor Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

triton-inference-server/server#8924 · Opened Aug 10, 2026
InferenceRequest.async_exec(decoupled=True) intermittently raises `RuntimeError: Invalid argument` under concurrent in-flight BLS to a decoupled model Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

triton-inference-server/server#8922 · 1 comment · Opened Aug 7, 2026
test: L0_lifecycle uses blind time.sleep(5) for model load/unload, causing flaky CI failures Open

Difficulty 3/5 1-2 days Newbie friendliness 76/100

triton-inference-server/server#8919 · Opened Aug 6, 2026
[vllm_backend] Switch to current engine to support the new multi modal models in vllm Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

triton-inference-server/server#8912 · Opened Jul 30, 2026
How to build a lite windows version? Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

triton-inference-server/server#8911 · Opened Jul 30, 2026
ONNXRuntime won't build Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

triton-inference-server/server#8906 · Opened Jul 28, 2026
[vLLM backend][Feature Request] Support vLLM-Omni for native audio/video input Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

triton-inference-server/server#8894 · 3 comments · 1 reaction · Opened Jul 21, 2026
Cuda SHM Module Cuda driver entry point API mismatch Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

triton-inference-server/server#8884 · 1 comment · Opened Jul 15, 2026
Intermittent server crash (`boost::interprocess::lock_exception` / `std::bad_alloc` / SIGSEGV) in Python backend under concurrent BLS load — 25.02 (2.55.0) Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

triton-inference-server/server#8882 · 3 comments · Opened Jul 14, 2026
Error verifying Inductor: torch._inductor.config.triton.autotune does not exist Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

triton-inference-server/server#8877 · Opened Jul 9, 2026
Dynamic-batcher throughput progressively degrades over time when `default_queue_policy.default_timeout_microseconds > 0` (waiting_consumer_count_ drift on payload merges) Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/server#8870 · 1 comment · Opened Jul 6, 2026
curl http://127.0.0.1:8000 Open

Difficulty 5/5 Over a week Newbie friendliness 10/100

triton-inference-server/server#8865 · Opened Jun 30, 2026
Feature Request: OpenAI-Compatible Frontend for Python backend Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/server#8862 · Opened Jun 29, 2026
Inaccuracies in PyTorch backend documentation Open

Documentation documentation issue

Difficulty 2/5 1-3 hours Newbie friendliness 65/100

triton-inference-server/server#8853 · 2 comments · Opened Jun 24, 2026
`json_schema` support for OpenAI Frontend for Structured Outputs Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

triton-inference-server/server#8852 · Opened Jun 24, 2026
Timeline for ONNX Runtime 1.26.0 support in Triton releases? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/server#8848 · Opened Jun 18, 2026
Iterative Sequences `force-end` at first state when use async execute Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

triton-inference-server/server#8843 · Opened Jun 16, 2026
Python stub aborted with glibc heap corruption, leading to 0 RPS on gRPC serving thread Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

triton-inference-server/server#8841 · 1 comment · Opened Jun 16, 2026
Python backend leaks disk: unpacked execution environments are never removed on model unload/reload Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

triton-inference-server/server#8840 · 2 comments · Opened Jun 15, 2026
Gradual queue-latency regime on TensorRT models with flat RSS and no GPU/host correlation — suspected scheduler-path lock contention Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/server#8839 · Opened Jun 15, 2026
Proposal: Verified Inference Path (DCC) for GPU Resource Protection Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

triton-inference-server/server#8838 · 1 comment · Opened Jun 13, 2026
Allow visibility of current trace/span inside the python backend Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

triton-inference-server/server#8837 · 1 reaction · Opened Jun 12, 2026
Update transformers because of warnings Open

Difficulty 2/5 1-3 hours Newbie friendliness 52/100

triton-inference-server/server#8824 · Opened Jun 9, 2026
Example: FunASR/SenseVoice speech recognition model on Triton Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

triton-inference-server/server#8813 · 1 comment · Opened Jun 2, 2026
Bug: cudnn_conv_algo_search silently ignored when configured via CUDA gpu_execution_accelerator Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

triton-inference-server/server#8808 · Opened May 30, 2026
[Compatibility Request] Support TensorRT-LLM v1.3 in Triton Inference Server containers Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

triton-inference-server/server#8805 · Opened May 29, 2026
[BUG] --http-header-forward-pattern has no effect on /generate endpoint (ensemble → Python backend) Open

triton-inference-server/server#8790 · 1 assignee · Opened May 19, 2026
Stuck in PENDING to EXECUTING state : full model Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

triton-inference-server/server#8781 · Opened May 14, 2026
Preprocessing in ensemble 2x slower than preprocessing on client side Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

triton-inference-server/server#8776 · Opened May 13, 2026
Python backend model missing model.py remains stuck in LOADING and cannot be unloaded Open

bug Python Backend

Difficulty 4/5 3-5 days Newbie friendliness 48/100

triton-inference-server/server#8766 · Opened May 5, 2026
SageMaker single-model endpoint returns 500 for empty-body POST to /invocations (should be 400) Open

Difficulty 3/5 1-2 days Newbie friendliness 58/100

triton-inference-server/server#8762 · 1 comment · Opened Apr 30, 2026
In-process Python API: `Server.stop()` hangs until `exit_timeout` and never invokes Python-backend `finalize()` Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

triton-inference-server/server#8755 · Opened Apr 22, 2026
Wrong input numbers/input spec for torch aoti export model Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 58/100

triton-inference-server/server#8754 · 1 comment · Opened Apr 22, 2026
Ensemble Model only marginally faster than separate single-model calls -- is this expected? Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

triton-inference-server/server#8740 · 2 comments · Opened Apr 15, 2026
vllm-omni support Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/server#8714 · Opened Mar 25, 2026
speculative config with vllm backend Open

triton-inference-server/server#8699 · 1 assignee · Opened Mar 13, 2026
Segmentation fault on model reload when using Python backend metrics due to shared Metric object across processes Open

bug metrics

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/server#8663 · 4 comments · Opened Feb 16, 2026
Segmentation fault on shutdown when using Python backend metrics Open

bug metrics

Difficulty 3/5 1-2 days Newbie friendliness 25/100

triton-inference-server/server#8662 · 4 comments · Opened Feb 16, 2026
triton+vllm serve embeddings but not support list[str] Open

Enhancement openai

Difficulty 4/5 3-5 days Newbie friendliness 30/100

triton-inference-server/server#8655 · Opened Feb 12, 2026
How to activate N-many TensorRT Optimization Profiles (TensorRT backend) Open

question

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/server#8629 · 4 comments · Opened Jan 30, 2026
Running async execute in the python backend the correct way Open

question

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/server#8615 · 2 comments · Opened Jan 21, 2026
Get stuck on the stage "Engine version 1.0.0 found in the config file, assuming engine(s) built by new builder API." Open

question TensorRT-LLM Backend

Difficulty 4/5 3-5 days Newbie friendliness 38/100

triton-inference-server/server#8540 · 2 comments · Opened Nov 24, 2025
Cannot remove future from background; KeyError: <Future at ... state=finished returned NoneType> Open

bug Python Backend

triton-inference-server/server#8537 · 12 comments · 1 assignee · Opened Nov 24, 2025
Honor the sampling flag in traceparent header for OpenTelemetry tracing Open

Feature

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/server#8474 · 2 comments · 2 reactions · Opened Oct 24, 2025
GitHub::Issue - build error Open

Build question

triton-inference-server/server#8462 · 26 comments · 1 assignee · Opened Oct 17, 2025

triton-inference-server/server

Get new issues in your inbox