Manager: stream handler defaultdicts grow unbounded across kernel lifecycle
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
## Objective
The four per-kernel stream defaultdicts (stream_pty_handlers, stream_execute_handlers, stream_proxy_handlers, stream_stdin_socks in src/ai/backend/manager/api/rest/stream/handler.py) never delete their outer keys. handle_kernel_terminating() uses a bare stream_stdin_socks[stream_key] access, so defaultdict.__missing__ inserts an empty key on every kernel termination, including kernels that never opened a stream.
## Evidence (live worker, Linux + memray)
- Real session create->terminate cycles left rest/stream/handler.py and defaultdict __missing__ frames in the memray --leaks (un-freed) set.
- Shutdown cleanup only iterates .values(); outer dict keys are never removed.
## Fix
- Use .get() instead of bare defaultdict access in handle_kernel_terminating.
- Delete all four per-kernel dict keys on kernel termination.
## Acceptance Criteria
- Per-kernel dict keys are removed on termination.
- Stream defaultdict sizes are bounded by the number of live kernels.
JIRA Issue: BA-6804
Contributor guide
Research direction
Start in src/ai/backend/manager/api/rest/stream/handler.py, inspecting the four per-kernel stream defaultdicts and handle_kernel_terminating(). Verify cleanup for kernels with and without opened streams; done means all four outer keys are removed on termination and their sizes remain bounded by live kernels.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100