lablup / lablup/backend.ai

Refactor kernel registry persistence from pickle to container-based recovery

Open
#6,333 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Problem

Currently, the agent uses pickle serialization to persist and restore the kernel registry (`kernel_registry`) between agent restarts. This approach has compatibility issues when Python versions or class structures change, potentially causing agent startup failures.

The pickle-based persistence is implemented in:

- `save_last_registry()`: Saves kernel_registry using `pickle.dump()` (line 3749)
- `scan_running_kernels()`: Loads kernel_registry using `pickle.load()` (line 2282)

## Goal

Replace pickle-based persistence with a more robust container metadata-based recovery mechanism that reads Docker container labels and port information to reconstruct kernel objects.

## Approach

1. Design an interface-based recovery system for flexibility
1. Maintain backward compatibility with pickle during transition
1. Implement container-based recovery using Docker labels and port mappings
1. Enable seamless migration between recovery strategies

## Benefits

- Improved compatibility across Python versions and code changes
- More resilient agent restart mechanism
- Better observability through container metadata
- Easier debugging and troubleshooting

JIRA Issue: BA-2771

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.