lablup / lablup/backend.ai

Introduce ReplicaGroup and resolve scaling/deploy race

Open
#11,849 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Background

The current deployment model has a race condition between scaling and deploy handlers because they operate on the same `desired_replica_count` field through separate locks (`LOCKID_DEPLOYMENT_AUTO_SCALER`, `LOCKID_DEPLOYMENT_CHECK_REPLICA`, `LOCKID_DEPLOYMENT_DEPLOYING`). Specifically:

- `_evaluate_deployment_scaling` selects scale-in candidates revision-blind, which can terminate routes belonging to a new revision being rolled out.
- `target_replica_count` carries two meanings (rollout final goal vs current revision's current goal) which the scaling and rolling handlers interpret differently.
- `Route.traffic_ratio` is stored but never applied to routing logic; it is fundamentally placed at the wrong level (per-route instead of per-group).
- Blue-Green deployment strategy raises `NotImplementedError`. Canary is not supported.
- The existing 2-pointer model (`current_revision_id`, `deploying_revision_id`) on the deployment cannot represent multiple parallel groups needed for canary or blue-green.

## Goals

1. Eliminate the race between scaling decisions and deploy progression by unifying them into a single mutator path.
2. Introduce `ReplicaGroup` as a first-class entity that owns the per-group `desired_replica_count`, `traffic_weight`, `current_revision_id`, `target_revision_id`, and lifecycle phase.
3. Limit each deployment to at most 2 groups, enforced structurally by reconciler step ordering (cleanup before creation), not by DB constraint or lock.
4. Add 5 sub_steps to the deploy axis: `INITIALIZING`, `PROVISIONING`, `PROMOTING`, `DRAINING`, `COMPLETED`. Each strategy uses these sub_steps differently (rolling skips DRAINING; canary/BG go through all 5).
5. Remove `Route.traffic_ratio` in favor of `ReplicaGroup.traffic_weight`.
6. Implement Blue-Green and Canary strategies on the new infrastructure.

## Non-goals

- New user-facing API design (no `/rollout` endpoint, etc.)
- UX changes
- Per-group coordinator (group state is handled within deployment-level reconciler)

## Approach

The work is split into 6 stories under this Epic, executed in dependency order:

1. Model setup (ReplicaGroup, Route.group_id, Deployment pointers, traffic_ratio removal, migration)
2. Deploy handler rewrite (5-stage sub_step, rolling on ReplicaGroup)
3. Scaling handler rewrite (absorb CheckReplica, revision-specific desired counts)
4. Reconcile handler refinement (invariant recovery)
5. Blue-Green strategy implementation
6. Canary strategy implementation

The core invariant — that each deployment has at most 2 ReplicaGroups, with `target_group_id` set only during rollouts — is maintained by the reconciler's fixed step order (cleanup → transition → initialize → strategy step → drift correction) within a deployment-level dispatch lock.

JIRA Issue: BA-6232

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.