huggingface/inference-benchmarker

Inference server benchmarking tool

Stars: 172
Forks: 34
Open beginner issues: 0
Indexed issues: 13

Dominant language: Rust
License: Apache-2.0
Last GitHub push: May 26, 2026
Latest indexed: Sep 17, 2026
Contributing guide: No contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

13 open issues indexed

Can i use local model's tokenizer and local dataset? Open

question

huggingface/inference-benchmarker#4 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
KeyError: 'model_name' while visualizing results Open

huggingface/inference-benchmarker#8 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature Request] Allow use of non OAI-like API url Open

huggingface/inference-benchmarker#9 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature Request] Publish results Open

huggingface/inference-benchmarker#10 · 3 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
multiple vllms with nginx Open

huggingface/inference-benchmarker#14 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
running with local tokenizer and dataset Open

huggingface/inference-benchmarker#22 · 0 comments · 5 reactions · 0 assignees · Indexed Sep 17, 2026
Index out of bounds error in `quantile_duration` in src/results.rs Open

huggingface/inference-benchmarker#23 · 2 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
Benchmark fails at the end and do not create json file test result Open

huggingface/inference-benchmarker#27 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Visualize the results doesn't work Open

huggingface/inference-benchmarker#28 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Benchmark throughput displayed is not the same than the one from vLLM Open

huggingface/inference-benchmarker#29 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
k8s deployment failure tgi libpython3.11.so.1.0: cannot open shared object file: No such file or directory Open

huggingface/inference-benchmarker#31 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Is it possible to use tekken.json instead of tokenizer.json? Open

huggingface/inference-benchmarker#38 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
thread 'tokio-runtime-worker' panicked at src/results.rs:252:31: index out of bounds: the len is 1 but the index is 1 Open

huggingface/inference-benchmarker#40 · 3 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026

A short digest of beginner-friendly GitHub issues.