vllm-project / vllm-project/aibrix
[Bug] Batch job `request_counts.failed` is always 0; failed requests are miscounted as `completed`
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
### 🐛 Describe the bug
## Component
`python/aibrix/aibrix/batch` — job progress tracking (`JobManager` / `JobMetaInfo`)
## Summary
When a batch request fails inference (after retries are exhausted), it is recorded
as a **completed** request instead of a **failed** one. `request_counts.failed`
stays `0` for the entire lifetime of every job, and `request_counts.completed`
is inflated by the number of failed requests.
This corrupts the user-facing request statistics (OpenAI Batch API exposes
`completed` / `failed` counts). It does **not** stall the job — finalization
still works — which is why it has gone unnoticed.
## Root cause
`JobMetaInfo.complete_one_request` supports a `failed` flag:
```python
# python/aibrix/aibrix/batch/job_manager.py:131
def complete_one_request(self, req_id, failed: bool = False):
...
elif not self._request_progress_bits[req_id]:
self.set_request_executed(req_id)
if failed:
self.status.request_counts.failed += 1 # never reached
else:
self.status.request_counts.completed += 1
```
But **none of the three call sites ever pass `failed=True`**:
- `job_manager.py:852` `mark_job_progress` → `complete_one_request(req_id)`
- `job_manager.py:877` `mark_jobs_progresses` → `complete_one_request(req_id)`
- `job_manager.py:909` `mark_job_progress_and_get_next_request` → `complete_one_request(req_id)`
### Steps to Reproduce
un a batch job where some requests deterministically fail inference (e.g. an endpoint that 500s for certain inputs). After the job finalizes:
Expected: request_counts.failed == .
Actual: request_counts.failed == 0, completed == total.
### Expected behavior
N/A
### Environment
nightly
Contributor guide
Research direction
Start in python/aibrix/aibrix/batch/job_manager.py, reading JobMetaInfo.complete_one_request and the three call sites in mark_job_progress, mark_jobs_progresses, and mark_job_progress_and_get_next_request. Trace how exhausted inference failures reach these paths. Done means failed requests increment request_counts.failed rather than completed, while successful requests and job finalization remain correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100