triton-inference-server/client

View on GitHub

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.

Stars: 701
Forks: 261
Open beginner issues: 0
Indexed issues: 61
Avg merge: 1d 1h
Merged PRs (30d): 9

Dominant language: C++
License: BSD-3-Clause
Last GitHub push: Sep 15, 2026
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

61 open issues indexed

Feature request: build an inference request once and reuse it across retries, without private APIs Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

triton-inference-server/client#918 · Opened Aug 25, 2026
Race condition at cuda shared memory module in python Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

triton-inference-server/client#914 · Opened Aug 9, 2026
Remove -Werror from cmake build configuration Open

Difficulty 1/5 Under an hour Newbie friendliness 84/100

triton-inference-server/client#902 · Opened Jun 12, 2026
Remove upper bound on grpcio so we don't have to build wheels locally in python 3.14? Open

Difficulty 2/5 1-3 hours Newbie friendliness 58/100

triton-inference-server/client#885 · 7 comments · 4 reactions · Opened Mar 31, 2026
Tests package included as top level package in python wheel Open

Difficulty 1/5 Under an hour Newbie friendliness 68/100

triton-inference-server/client#876 · Opened Jan 27, 2026
How to install tritonclient on an Apple M type CPU? Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

triton-inference-server/client#870 · Opened Dec 4, 2025
tritonclient[grpc] is pinned to protobuf 5.x Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

triton-inference-server/client#869 · 1 comment · 5 reactions · Opened Nov 27, 2025
Request for a tag or release Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

triton-inference-server/client#864 · 1 reaction · Opened Nov 14, 2025
Are there any plans to provide C# examples? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/client#861 · 1 comment · Opened Nov 10, 2025
Python package install misplaces a license file outside of tritonclient directories Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

triton-inference-server/client#858 · Opened Oct 22, 2025
tritonclient version 2.61.0 not installing on macOS with UV due to perf-analyzer not having macOS arm64 wheels Open

triton-inference-server/client#856 · 11 comments · 13 reactions · 2 assignees · Opened Oct 16, 2025
Support free-threaded CPython (3.14t) Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/client#855 · Opened Oct 16, 2025
[BUG] TritonClient.grpc.InferResult get_output() does not return properly Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

triton-inference-server/client#850 · Opened Sep 22, 2025
[Feature Request] Add gzip compression options for Java client send/receive Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/client#845 · Opened Aug 5, 2025
link grpcclient_static error. undefined reference to grpc Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/client#843 · Opened Jul 17, 2025
error parsing results in genai-perf when sending to dynamo workload Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/client#840 · Opened Jul 3, 2025
onnx model inference bug Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/client#839 · Opened Jun 23, 2025
[Question] Is this lib suppose to be build with the 3rd party curl version and not system version ? Open

Difficulty 2/5 1-3 hours Newbie friendliness 25/100

triton-inference-server/client#838 · Opened May 27, 2025
DLPack tensor is not contiguous. Only contiguous DLPack tensors that are stored in C-Order are supported. Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/client#834 · 1 comment · Opened Apr 20, 2025
Cannot obtain server metadata (C++ API) Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/client#832 · 1 comment · Opened Apr 17, 2025
grpcio version Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

triton-inference-server/client#831 · Opened Mar 31, 2025
Get model config without loading Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

triton-inference-server/client#828 · 1 comment · Opened Mar 25, 2025
Windows C++ Build: vcpkg's rapidjson not properly linked Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/client#825 · Opened Feb 27, 2025
How do I pass control input to python-stateful backend? Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/client#824 · Opened Feb 21, 2025
[Feature Request] Official Rust Client Library for Triton Inference Server Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

triton-inference-server/client#823 · 9 reactions · Opened Feb 21, 2025
Sticky Load Balancing during inference not hitting all the Triton Servers sitting behind the Load Balancer Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/client#820 · 1 comment · Opened Feb 13, 2025
[Feature Request] Support using system versions of gRPC, Protobuf, Abseil in CMake Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/client#819 · 1 reaction · Opened Jan 19, 2025
[Question] Help with Client-Side Batching for Large Requests in Triton Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

triton-inference-server/client#818 · Opened Jan 17, 2025
Read back data written in cuda shared memory in python Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/client#816 · Opened Dec 16, 2024
Performance Discrepancy Between Triton Client SDK and perf_analyzer Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

triton-inference-server/client#815 · 2 comments · 1 reaction · Opened Dec 10, 2024
InferenceServerHttpClient::Create() failed with error: std::bad_alloc Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/client#806 · Opened Nov 13, 2024
Incorrect grpcio version Python client Open

Difficulty 1/5 Under an hour Newbie friendliness 45/100

triton-inference-server/client#804 · Opened Nov 8, 2024
[GRPC_CLIENT] LoadModel with serialized model raise an error: "Poll failed for model directory" on native windows triton. Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/client#803 · Opened Nov 6, 2024
The dependency information of the Python package needs to be updated Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

triton-inference-server/client#796 · 1 reaction · Opened Oct 16, 2024
Potential memory leak Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/client#791 · 2 comments · Opened Oct 8, 2024
Tritonclient CUDA shared memory set_shared_memory_region_from_dlpack fails with non-set stream for Torch tensor. Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

triton-inference-server/client#789 · Opened Oct 5, 2024
genai-perf exception offline-mode Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

triton-inference-server/client#785 · Opened Sep 24, 2024
原始是byte[]的数据，没法传。会报triton.client.InferenceException: [request id: <id_unknown>] inference input data-type is 'INT8', model expects 'BYTES' for 'xxxxx' Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/client#784 · 3 comments · Opened Sep 24, 2024
Re-use client or create new one each time? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

triton-inference-server/client#783 · Opened Sep 19, 2024
KV-cache of Tensorrt_llm Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

triton-inference-server/client#782 · Opened Sep 16, 2024
Use try-with-resources for InferenceServerClient to handle resource management and potential TimeoutException Open

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

triton-inference-server/client#780 · Opened Sep 7, 2024
tensorrtllm and vllm backend results are different using genai-perf Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/client#779 · 1 comment · Opened Sep 5, 2024
Unexpected Behavior: ModelInferRequest Fields Overwritten with Incorrect Values in Triton C++ Client Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

triton-inference-server/client#778 · Opened Sep 5, 2024
Failing with Generic Error message: Failed to obtain stable measurement. Open

triton-inference-server/client#777 · 11 comments · 1 assignee · Opened Aug 20, 2024
Allow client to request subset of ensemble model outputs Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/client#763 · 2 comments · Opened Jul 24, 2024
Decreased Accuracy in Text Detection and Recognition Models after Upgrading to tritonclient 23.04-py3 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/client#738 · Opened Jul 8, 2024
Benchmarking VQA Model with Large Base64-Encoded Input Using perf_analyzer Open

question

Difficulty 4/5 3-5 days Newbie friendliness 25/100

triton-inference-server/client#736 · 2 comments · Opened Jul 5, 2024
Memory leak from grpcio Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

triton-inference-server/client#731 · 1 comment · Opened Jul 2, 2024
Bug in grpc client Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

triton-inference-server/client#726 · Opened Jun 28, 2024
How to write cmakelist to use grpcclient? Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

triton-inference-server/client#714 · Opened Jun 26, 2024

triton-inference-server/client

Get new issues in your inbox