v2 deployment DTO always returns empty replica_state.replica_ids
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
## Summary
The v2 deployment API returns `replica_state.replica_ids = []` for **every** deployment — the field is hard-coded empty and never populated from the deployment's routes/replicas. A deployment that is `READY` with a running, traffic-active route still reports zero replica ids.
## Where
`src/ai/backend/manager/services/deployment/service.py` — both DTO converters hard-code it:
```python
# _convert_deployment_info_to_data() — line 223-226
replica_state=ReplicaStateData(
desired_replica_count=_deployment_desired_replica_count(info),
replica_ids=[], # Not available in DeploymentInfo
),
# _convert_deployment_info_to_legacy_data() — line 257-260
replica_state=ReplicaStateData(
desired_replica_count=_deployment_desired_replica_count(info),
replica_ids=[], # Not available in DeploymentInfo
),
```
The root reason is upstream: `EndpointRow.to_modern_deployment_info()` (`src/ai/backend/manager/models/endpoint/row.py:798`) builds `DeploymentInfo` **without loading routes** — it eagerly loads replica groups, but only to read revision ids — and `DeploymentRepository.get_endpoint_info()` (`src/ai/backend/manager/repositories/deployment/repository.py:350`) doesn't either, hence the "Not available in DeploymentInfo" comment. The API adapters (`src/ai/backend/manager/api/adapters/deployment/adapter.py:2343`, `src/ai/backend/manager/api/rest/deployment/adapter.py:162`) just pass the empty list through.
This also contradicts the DTO's own contract: `ReplicaStateInfo.replica_ids` is documented as `"IDs of current replicas"` (`src/ai/backend/common/dto/manager/deployment/response.py:103`).
Confirmed on `26.4.6`, and re-confirmed on `main` at `763d07f49f` (2026-09-10).
## Why it matters
`replica_ids` is the only field on the REST deployment DTO that tells a client whether a `READY` deployment's replicas have actually come up. `READY` alone only means the rollout settled on the current revision — it does not assert any replica is serving. Consumers that gate on "are replicas provisioned?" by reading `len(replica_state.replica_ids)` therefore always see **zero** and conclude the deployment is not serving, even when it is healthy and routing traffic.
Concretely, this broke Backend.AI FastTrack: a model-serving pipeline task gated readiness on `len(replica_ids) >= desired_replica_count` and so was pinned in `PREPARING` forever (then failed at retry-timeout) for endpoints that were in fact serving. (Worked around there by deriving readiness from `search_routes()` instead — lablup/backend.ai-fasttrack#4781 / #4782 — but the DTO contract is still wrong for every other consumer.)
GraphQL clients are not affected in the same way: the `Deployment` type now exposes a `replicas` connection (`src/ai/backend/manager/api/gql/deployment/types/deployment.py:488`) that lists them directly. REST clients have no equivalent on the deployment DTO.
## Reproduction
1. Create a single-replica model deployment; let it reach `READY` with a running route.
2. `GET` the deployment (v2) or call the deployment service `get_deployment_by_id`.
3. Observe `status == READY`, a route with `status=running, traffic_status=active`, but `replica_state.replica_ids == []`.
## Proposed fix
Pick one of the following; either way, stop returning a field that silently lies.
**Option A — populate it.** Routes **are** the replicas (`RoutingRow.id` is a `ReplicaID`, and the route observer already does exactly this in `src/ai/backend/manager/sokovan/deployment/route/handlers/observer/health_check.py:79`: `replica_ids = [r.route_id for r in routes]`).
1. Load the deployment's routes in `EndpointRow.to_modern_deployment_info()` / `get_endpoint_info()` (or fetch them in the service before conversion).
2. Filter to active routes (`RouteStatus.active_route_statuses()` → `{PROVISIONING, RUNNING}`; decide whether `traffic_status == ACTIVE` should be required).
3. Set `replica_ids = [route.route_id for route in active_routes]` in both `_convert_deployment_info_to_data` and `_convert_deployment_info_to_legacy_data`.
Cost: a route load on every deployment read, including list endpoints.
**Option B — deprecate it.** Mark `replica_state.replica_ids` deprecated in both REST DTOs (v1 `response.py:103`, v2 `common/dto/manager/v2/deployment/types.py:139`), point clients to the replica/route listing (`search_replicas`, or the GraphQL `Deployment.replicas` connection), and drop the hard-coded converter lines once the deprecation window passes. No extra query per read, but REST clients need a second call to get replica state.
For Option A, decide explicitly whether `replica_ids` means "all current replicas" (PROVISIONING + RUNNING) or "serving replicas" (RUNNING only) and document it on `ReplicaStateInfo.replica_ids`, since clients gate on the count.
Contributor guide
Research direction
Start with the two converters in src/ai/backend/manager/services/deployment/service.py, then trace how EndpointRow.to_modern_deployment_info() and DeploymentRepository.get_endpoint_info() build DeploymentInfo. Reproduce the empty replica_ids response with a READY deployment and inspect the route-loading behavior. Before coding, get a decision on populating active route IDs versus deprecating the field; done means both DTO paths and their documented contract consistently reflect that choice.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100