lablup / lablup/backend.ai

Enable auto-scaling rules to remain active during rolling update deployment

Open
#10,301 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Background

Currently, auto-scaling (CheckReplicaHandler) only runs in READY state and rolling update (DeployingProvisioningHandler) only runs in DEPLOYING state. These two are mutually exclusive by lifecycle. The goal is to allow auto-scaling rules to continue evaluating and updating `desired_replica_count` while a rolling update is in progress, so that the rolling update FSM naturally adapts to the changing target.

## Current Architecture

- `CheckReplicaHandler` targets READY only → calculates `desired_replica_count` → transitions to SCALING
- `ScalingHandler` targets SCALING only → creates/terminates routes → transitions to READY
- `DeployingProvisioningHandler` targets DEPLOYING → runs rolling update FSM each cycle
- Rolling update FSM already reads `target_replica_count` fresh each cycle (rolling_update.py:82), so it can naturally adapt to changed values

## Why It Already Almost Works

The rolling update FSM in `RollingUpdateStrategy.evaluate_cycle()` reads `deployment.replica_spec.target_replica_count` every cycle. If auto-scaling updates `desired_replica_count` in DB during DEPLOYING, the next FSM cycle would pick up the new value and adjust surge/unavailability budgets automatically:

- **Scale up (4→6):** still_needed increases, FSM creates more new-revision routes
- **Scale down (6→4):** still_needed decreases, FSM drains more old-revision routes, fewer new-revision routes needed

## Required Work

### 1. Allow auto-scaling rule evaluation during DEPLOYING

CheckReplicaHandler currently targets only READY state. It needs to also evaluate deployments in DEPLOYING state, but only for the purpose of updating `desired_replica_count` — NOT triggering a SCALING transition. The scaling action itself should be left to the rolling update FSM.

- Option A: Extend CheckReplicaHandler to target both READY and DEPLOYING, with different transition logic (READY → SCALING for READY state, no lifecycle change for DEPLOYING state — just update desired_replica_count in DB)
- Option B: Extract the desired_replica_count calculation into a separate handler that runs regardless of lifecycle state, while keeping the SCALING transition only for READY state

### 2. Guard route eviction for deploying-revision routes

RouteEvictionHandler operates independently of deployment lifecycle. During DEPLOYING, new-revision routes that are temporarily UNHEALTHY (warming up) could be evicted. The eviction handler should be aware of the deploying revision and avoid evicting new-revision routes that are still in the warm-up window.

### 3. Handle scale-down below current new-revision route count

If auto-scaling scales down from 6 to 2 during a rolling update that already has 4 healthy new-revision routes, the FSM currently has no mechanism to terminate excess new-revision routes — it only drains old-revision routes. The FSM needs logic to handle `new_healthy > desired` by draining excess new-revision routes as well.

### 4. Completion condition with dynamic desired count

The current completion check is: `old_active == 0 AND new_healthy >= desired`. With a changing desired count, verify this still works correctly. For example, if desired drops from 6 to 4 and we have 4 new_healthy + 2 old_active, the FSM should drain the old routes and complete — not get stuck.

### 5. Observability

Log when the FSM detects a desired count change mid-deployment. This helps operators understand why the rolling update changed pace.

## Key Files

- `sokovan/deployment/strategy/rolling_update.py` — FSM: evaluate_cycle, _compute_routes_to_create/terminate
- `sokovan/deployment/handlers/replica.py` — CheckReplicaHandler (needs DEPLOYING support)
- `sokovan/deployment/executor.py` — calculate_desired_replicas, scale_deployment
- `sokovan/deployment/route/handlers/route_eviction.py` — RouteEvictionHandler (needs deploying-revision awareness)
- `data/deployment/types.py` — ReplicaSpec.target_replica_count

## Acceptance Criteria

- Auto-scaling rules continue to evaluate and update desired_replica_count during DEPLOYING state
- Rolling update FSM naturally adapts to changed desired count each cycle (already works by design)
- Scale-down during deployment correctly drains excess new-revision routes when new_healthy > desired
- Route eviction does not interfere with new-revision routes during warm-up
- Deployment completes correctly regardless of desired count changes mid-deployment

JIRA Issue: BA-5290

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.