lablup / lablup/backend.ai

Manage route status during inference session restart

Open
#9,810 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

Keep the route alive during inference session container restart, transitioning route to DEGRADED during restart and recovering to HEALTHY after successful restart.

Current behavior: container crash → session TERMINATED → route TERMINATED → new route created (replica guarantee).
Target: container crash → session RESTARTING → route DEGRADED → session RUNNING → route health check → HEALTHY.

Key files:
- manager/registry.py (restart_session - add route status update)
- manager/models/routing/row.py (RoutingRow status update)
- manager/data/deployment/types.py (RouteStatus enum)
- manager/sokovan/deployment/route/handlers/health_check.py (health check recovery)

## Success Criteria
- [ ] Route transitions to DEGRADED when session enters RESTARTING state
- [ ] Route does NOT transition to TERMINATED during restart
- [ ] After successful restart (session RUNNING), route health check recovers route to HEALTHY
- [ ] Route error_data records the restart event
- [ ] pants check passes for affected packages

JIRA Issue: BA-4958

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.