lablup / lablup/backend.ai

Manager: stream handler defaultdicts grow unbounded across kernel lifecycle

Open Beginner friendly
#12,693 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

## Objective

The four per-kernel stream defaultdicts (stream_pty_handlers, stream_execute_handlers, stream_proxy_handlers, stream_stdin_socks in src/ai/backend/manager/api/rest/stream/handler.py) never delete their outer keys. handle_kernel_terminating() uses a bare stream_stdin_socks[stream_key] access, so defaultdict.__missing__ inserts an empty key on every kernel termination, including kernels that never opened a stream.

## Evidence (live worker, Linux + memray)

- Real session create->terminate cycles left rest/stream/handler.py and defaultdict __missing__ frames in the memray --leaks (un-freed) set.
- Shutdown cleanup only iterates .values(); outer dict keys are never removed.

## Fix

- Use .get() instead of bare defaultdict access in handle_kernel_terminating.
- Delete all four per-kernel dict keys on kernel termination.

## Acceptance Criteria

- Per-kernel dict keys are removed on termination.
- Stream defaultdict sizes are bounded by the number of live kernels.

JIRA Issue: BA-6804

Contributor guide

Open the contributing guide

Research direction

Start in src/ai/backend/manager/api/rest/stream/handler.py, inspecting the four per-kernel stream defaultdicts and handle_kernel_terminating(). Verify cleanup for kernels with and without opened streams; done means all four outer keys are removed on termination and their sizes remain bounded by live kernels.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.