huggingface / huggingface/inference-benchmarker

Index out of bounds error in `quantile_duration` in src/results.rs

Open
#23 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
172
Forks
34
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
An "index out of bounds" error occurs in `src/results.rs` during benchmark runs. The error message `index out of bounds: the len is 1 but the index is 1` at `src/results.rs:252:31` points to an issue in the `quantile_duration` function.

**To Reproduce**
Steps to reproduce the behavior:
1. Run a benchmark with settings that might result in only one successful response.
2. Observe the panic.

**Expected behavior**
The `quantile_duration` function should handle cases where the `data` vector has only one element, possibly by returning that single element or a predefined value, instead of panicking.

**Console logs**
```
export URL = http://localhost:8000/v1
sudo docker run --network host \
-e HF_TOKEN=$HF_TOKEN \
-v ~/inference-benchmarker-results:/opt/inference-benchmarker/results \
inference_benchmarker inference-benchmarker \
--prompt-options "num_tokens=200,max_tokens=220,min_tokens=180,variance=10" \
--decode-options "num_tokens=200,max_tokens=220,min_tokens=180,variance=10"
--url $URL
...
thread 'tokio-runtime-worker' panicked at /usr/local/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ratatui-0.29.0/src/terminal/init.rs:52:16:
thread 'tokio-runtime-worker' panicked at src/results.rs:252:31:chmarker/src/event.rs:40:48:cf8c6b5b557f/ratatui-0.29.0/src/terminal/init
index out of bounds: the len is 1 but the index is 1ailed to initialize input reader" }
```

**Possible Cause**
The `quantile_duration` function at `src/results.rs:252` accesses `data[i as usize + 1]` without checking if `i as usize + 1` is a valid index. If `data` has only one element, and `i` is 0 (which can happen if `quantile` is such that `(data.len() - 1) as f64 * quantile` is less than 1), then `i as usize + 1` would be 1, leading to an out-of-bounds access.

**Suggested Fix**
Add a check within the `quantile_duration` function to handle cases where the length of the `data` vector is 1, or when `i as usize + 1` would be out of bounds. For example, if `data.len()` is 1, the function could return `data[0].as_secs_f64()`.

**Environment:**
- OS: Ubuntu 24 LTS as WSL with Windows 11 host
- inference-benchmarker version: build `inference_benchmarker:latest` from `2025-05-15T18:48:42.880472539+02:00`

**Remark**
this issue was created with the help of Gemini 2.5 Pro (especially the parts about possible cause and suggested fix...)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.