Lightning-AI / Lightning-AI/LitServe
Failed requests leave active_requests permanently incremented
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.9k
- Forks
- 304
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 6
Description
## Bug
With `track_requests=True`, `RequestCountMiddleware` increments the shared counter before dispatch and decrements it only after the downstream app returns. If request handling raises, the decrement is skipped. Each 500 response can therefore leave a phantom active request in `LitServer.active_requests`, affecting callbacks and monitoring that rely on the count.
## Reproduction
On `main` at `16248473bc3127bef5d2cfe11de9d9677d9ee2c3`:
```python
import multiprocessing as mp
from fastapi import FastAPI
from fastapi.testclient import TestClient
from litserve.middlewares import RequestCountMiddleware
counter = mp.Value("i", 0, lock=True)
app = FastAPI()
app.add_middleware(RequestCountMiddleware, active_counter=counter)
@app.get("/fail")
def fail():
raise RuntimeError("boom")
with TestClient(app, raise_server_exceptions=False) as client:
assert client.get("/fail").status_code == 500
assert counter.value == 0 # currently 1
```
## Expected behavior
The counter is decremented when downstream request handling exits, whether it returns normally or raises, while preserving the original exception behavior.
This report was prepared with Codex assistance.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating RequestCountMiddleware and the active_counter handling described in the issue. Reproduce the failure with FastAPI's TestClient and a raising endpoint, then add a regression test showing the counter returns to zero after an exception while the original 500 behavior is preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fastapi, python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 78/100