Webserver-to-manager client pool can close sessions mid-stream (same defect as #13419)
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
Follow-up to #13419 / #13425.
## Summary
#13425 adds `ClientPool(keep_inflight_sessions=...)` and wires it only into the App Proxy worker (`proxy_worker.client_pool_keep_inflight_sessions`). The webserver's pool toward the manager has the identical pattern and is not covered.
## Where
`src/ai/backend/web/server.py` — `ClientPool` with `auto_decompress=False`, default 600s cleanup interval, proxying streamed manager responses. A streamed response longer than 600s on an otherwise idle `ClientKey` has its session closed underneath it; the read then stalls silently rather than raising.
Other `ClientPool` constructions (manager `registry.py`, `sokovan.py`, `notification_center.py`, `prometheus_client.py`, appproxy `worker/server.py`, storage `hammerspace/client.py`) are non-streaming and lower risk.
## Proposal
- Expose a webserver config flag mirroring `client_pool_keep_inflight_sessions` and pass it to the pool.
- Decide whether the guard should become the default once it has run in production on the App Proxy worker.
Contributor guide
Research direction
Start in src/ai/backend/web/server.py at the webserver ClientPool construction, then compare it with the App Proxy worker wiring for client_pool_keep_inflight_sessions. Trace how the webserver configuration reaches that pool and verify that a streamed manager response remains usable beyond the 600-second cleanup interval; done means the flag is exposed and passed through, with its default decision documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100