Lightning-AI / Lightning-AI/LitServe
Potential Memory Leak in Response Buffer
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.9k
- Forks
- 304
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 6
Description
## 🐛 Bug
The [response_buffer](https://github.com/Lightning-AI/LitServe/blob/08a9caa4360aeef94ee585fc5e88f721550d267b/src/litserve/server.py#L74) dictionary could grow indefinitely if requests fail before their responses are processed.
Implement a cleanup mechanism or timeout for orphaned entries
#### Code sample
### Expected behavior
### Environment
If you published a Studio with your bug report, we can automatically get this information. Otherwise, please describe:
- PyTorch/Jax/Tensorflow Version (e.g., 1.0):
- OS (e.g., Linux):
- How you installed PyTorch (`conda`, `pip`, source):
- Build command you used (if compiling from source):
- Python version:
- CUDA/cuDNN version:
- GPU models and configuration:
- Any other relevant information:
### Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/litserve/server.py around the response_buffer definition at line 74, then trace how request failures and response processing affect its entries. Determine an appropriate cleanup mechanism or timeout for orphaned entries; the issue is complete when failed requests no longer allow the dictionary to grow indefinitely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100