vllm-project / vllm-project/aibrix

[Bug] Batch job `request_counts.failed` is always 0; failed requests are miscounted as `completed`

Open Beginner friendly
#2,253 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

### 🐛 Describe the bug

## Component
`python/aibrix/aibrix/batch` — job progress tracking (`JobManager` / `JobMetaInfo`)

## Summary
When a batch request fails inference (after retries are exhausted), it is recorded
as a **completed** request instead of a **failed** one. `request_counts.failed`
stays `0` for the entire lifetime of every job, and `request_counts.completed`
is inflated by the number of failed requests.

This corrupts the user-facing request statistics (OpenAI Batch API exposes
`completed` / `failed` counts). It does **not** stall the job — finalization
still works — which is why it has gone unnoticed.

## Root cause
`JobMetaInfo.complete_one_request` supports a `failed` flag:

```python
# python/aibrix/aibrix/batch/job_manager.py:131
def complete_one_request(self, req_id, failed: bool = False):
...
elif not self._request_progress_bits[req_id]:
self.set_request_executed(req_id)
if failed:
self.status.request_counts.failed += 1 # never reached
else:
self.status.request_counts.completed += 1
```

But **none of the three call sites ever pass `failed=True`**:

- `job_manager.py:852` `mark_job_progress` → `complete_one_request(req_id)`
- `job_manager.py:877` `mark_jobs_progresses` → `complete_one_request(req_id)`
- `job_manager.py:909` `mark_job_progress_and_get_next_request` → `complete_one_request(req_id)`

### Steps to Reproduce

un a batch job where some requests deterministically fail inference (e.g. an endpoint that 500s for certain inputs). After the job finalizes:

Expected: request_counts.failed == .
Actual: request_counts.failed == 0, completed == total.

### Expected behavior

N/A

### Environment

nightly

Contributor guide

Open the contributing guide

Research direction

Start in python/aibrix/aibrix/batch/job_manager.py, reading JobMetaInfo.complete_one_request and the three call sites in mark_job_progress, mark_jobs_progresses, and mark_job_progress_and_get_next_request. Trace how exhausted inference failures reach these paths. Done means failed requests increment request_counts.failed rather than completed, while successful requests and job finalization remain correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.