lablup / lablup/backend.ai

Separate Endpoint status into Traffic Status and Lifecycle Status

Open
#8,116 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Motivation

The current `EndpointStatus` enum conflates two distinct concepts - deployment lifecycle states and traffic health states. This creates confusion when an endpoint's lifecycle is complete but traffic health varies, making it difficult to provide clear status semantics to API consumers.

- Mixed status like `DESTROYING` (lifecycle) alongside `HEALTHY`/`UNHEALTHY` (traffic) in the same enum
- Difficult to represent scenarios where lifecycle is `READY` but traffic is `DEGRADED`
- API consumers cannot distinguish between deployment progress and service health

## Objective

Refactor the Endpoint status model to clearly separate **traffic/health status** from **deployment lifecycle status**, providing better observability and clearer API semantics.

- Define distinct status enums for lifecycle and traffic concerns
- Enable independent tracking of deployment state vs. service health
- Maintain backward compatibility with existing API consumers

## Details

### Current Status Definitions

**EndpointStatus** (mixed):

- `READY`, `PROVISIONING` (deprecated), `HEALTHY`, `UNHEALTHY`, `DESTROYING`, `DESTROYED`, `DEGRADED`

**EndpointLifecycle** (lifecycle-focused):

- `PENDING`, `CREATED` (deprecated), `SCALING`, `READY`, `DEPLOYING`, `DESTROYING`, `DESTROYED`

**RouteStatus** (per-route health):

- `PROVISIONING`, `HEALTHY`, `UNHEALTHY`, `DEGRADED`, `TERMINATING`, `TERMINATED`, `FAILED_TO_START`

### Proposed Solution

1. **EndpointLifecycleStatus** - for deployment lifecycle states:
- `PENDING` - Waiting for initial provisioning
- `DEPLOYING` - Creating/updating routes
- `READY` - Deployment is complete
- `SCALING` - Adjusting replica count
- `DESTROYING` - Being removed
- `DESTROYED` - Fully removed
1. **EndpointTrafficStatus** - for aggregated traffic/health status:
- `HEALTHY` - All routes are healthy
- `UNHEALTHY` - All routes are unhealthy or no healthy routes
- `DEGRADED` - Some routes healthy, some not (partial availability)
- `NO_TRAFFIC` - No routes available

### Implementation Tasks

1. Define new `EndpointTrafficStatus` enum
1. Add `traffic_status` field to `EndpointRow` or compute it from route statuses
1. Update GraphQL schema to expose both `lifecycle_stage` and `traffic_status`
1. Update API responses to include both status fields
1. Implement traffic status computation logic (aggregate from `RouteStatus`)
1. Update status transition handlers in Sokovan deployment handlers
1. Add backward-compatible `status` field mapping
1. Update documentation and API specs
1. Add unit tests for status computation logic

### Related Files

- `src/ai/backend/common/data/endpoint/types.py`
- `src/ai/backend/manager/models/endpoint/row.py`
- `src/ai/backend/manager/api/gql_legacy/endpoint.py`
- `src/ai/backend/manager/data/deployment/types.py`
- `src/ai/backend/manager/sokovan/deployment/handlers/`

## Impact

- GraphQL API: New `traffic_status` field exposed alongside existing `lifecycle_stage`
- REST API: Response payload changes to include both status fields
- Frontend/WebUI: May need updates to display separated status information
- Backward compatibility maintained via deprecated combined `status` field

JIRA Issue: BA-3939

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.