Separate Endpoint status into Traffic Status and Lifecycle Status
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
## Motivation
The current `EndpointStatus` enum conflates two distinct concepts - deployment lifecycle states and traffic health states. This creates confusion when an endpoint's lifecycle is complete but traffic health varies, making it difficult to provide clear status semantics to API consumers.
- Mixed status like `DESTROYING` (lifecycle) alongside `HEALTHY`/`UNHEALTHY` (traffic) in the same enum
- Difficult to represent scenarios where lifecycle is `READY` but traffic is `DEGRADED`
- API consumers cannot distinguish between deployment progress and service health
## Objective
Refactor the Endpoint status model to clearly separate **traffic/health status** from **deployment lifecycle status**, providing better observability and clearer API semantics.
- Define distinct status enums for lifecycle and traffic concerns
- Enable independent tracking of deployment state vs. service health
- Maintain backward compatibility with existing API consumers
## Details
### Current Status Definitions
**EndpointStatus** (mixed):
- `READY`, `PROVISIONING` (deprecated), `HEALTHY`, `UNHEALTHY`, `DESTROYING`, `DESTROYED`, `DEGRADED`
**EndpointLifecycle** (lifecycle-focused):
- `PENDING`, `CREATED` (deprecated), `SCALING`, `READY`, `DEPLOYING`, `DESTROYING`, `DESTROYED`
**RouteStatus** (per-route health):
- `PROVISIONING`, `HEALTHY`, `UNHEALTHY`, `DEGRADED`, `TERMINATING`, `TERMINATED`, `FAILED_TO_START`
### Proposed Solution
1. **EndpointLifecycleStatus** - for deployment lifecycle states:
- `PENDING` - Waiting for initial provisioning
- `DEPLOYING` - Creating/updating routes
- `READY` - Deployment is complete
- `SCALING` - Adjusting replica count
- `DESTROYING` - Being removed
- `DESTROYED` - Fully removed
1. **EndpointTrafficStatus** - for aggregated traffic/health status:
- `HEALTHY` - All routes are healthy
- `UNHEALTHY` - All routes are unhealthy or no healthy routes
- `DEGRADED` - Some routes healthy, some not (partial availability)
- `NO_TRAFFIC` - No routes available
### Implementation Tasks
1. Define new `EndpointTrafficStatus` enum
1. Add `traffic_status` field to `EndpointRow` or compute it from route statuses
1. Update GraphQL schema to expose both `lifecycle_stage` and `traffic_status`
1. Update API responses to include both status fields
1. Implement traffic status computation logic (aggregate from `RouteStatus`)
1. Update status transition handlers in Sokovan deployment handlers
1. Add backward-compatible `status` field mapping
1. Update documentation and API specs
1. Add unit tests for status computation logic
### Related Files
- `src/ai/backend/common/data/endpoint/types.py`
- `src/ai/backend/manager/models/endpoint/row.py`
- `src/ai/backend/manager/api/gql_legacy/endpoint.py`
- `src/ai/backend/manager/data/deployment/types.py`
- `src/ai/backend/manager/sokovan/deployment/handlers/`
## Impact
- GraphQL API: New `traffic_status` field exposed alongside existing `lifecycle_stage`
- REST API: Response payload changes to include both status fields
- Frontend/WebUI: May need updates to display separated status information
- Backward compatibility maintained via deprecated combined `status` field
JIRA Issue: BA-3939
Contributor guide
Assessment
This issue has not been assessed yet.