Lightning-AI / Lightning-AI/LitServe

Failed requests leave active_requests permanently incremented

Open Beginner friendly
#747 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.9k
Forks
304
Avg merge
3d 13h
Merged PRs (30d)
6

Description

## Bug

With `track_requests=True`, `RequestCountMiddleware` increments the shared counter before dispatch and decrements it only after the downstream app returns. If request handling raises, the decrement is skipped. Each 500 response can therefore leave a phantom active request in `LitServer.active_requests`, affecting callbacks and monitoring that rely on the count.

## Reproduction

On `main` at `16248473bc3127bef5d2cfe11de9d9677d9ee2c3`:

```python
import multiprocessing as mp
from fastapi import FastAPI
from fastapi.testclient import TestClient
from litserve.middlewares import RequestCountMiddleware

counter = mp.Value("i", 0, lock=True)
app = FastAPI()
app.add_middleware(RequestCountMiddleware, active_counter=counter)

@app.get("/fail")
def fail():
raise RuntimeError("boom")

with TestClient(app, raise_server_exceptions=False) as client:
assert client.get("/fail").status_code == 500

assert counter.value == 0 # currently 1
```

## Expected behavior

The counter is decremented when downstream request handling exits, whether it returns normally or raises, while preserving the original exception behavior.

This report was prepared with Codex assistance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating RequestCountMiddleware and the active_counter handling described in the issue. Reproduce the failure with FastAPI's TestClient and a raising endpoint, then add a regression test showing the counter returns to zero after an exception while the original 500 behavior is preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, python
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.