lablup / lablup/backend.ai

v2 deployment DTO always returns empty replica_state.replica_ids

Open
#12,440 0 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

## Summary

The v2 deployment API returns `replica_state.replica_ids = []` for **every** deployment — the field is hard-coded empty and never populated from the deployment's routes/replicas. A deployment that is `READY` with a running, traffic-active route still reports zero replica ids.

## Where

`src/ai/backend/manager/services/deployment/service.py` — both DTO converters hard-code it:

```python
# _convert_deployment_info_to_data() — line 223-226
replica_state=ReplicaStateData(
desired_replica_count=_deployment_desired_replica_count(info),
replica_ids=[], # Not available in DeploymentInfo
),
# _convert_deployment_info_to_legacy_data() — line 257-260
replica_state=ReplicaStateData(
desired_replica_count=_deployment_desired_replica_count(info),
replica_ids=[], # Not available in DeploymentInfo
),
```

The root reason is upstream: `EndpointRow.to_modern_deployment_info()` (`src/ai/backend/manager/models/endpoint/row.py:798`) builds `DeploymentInfo` **without loading routes** — it eagerly loads replica groups, but only to read revision ids — and `DeploymentRepository.get_endpoint_info()` (`src/ai/backend/manager/repositories/deployment/repository.py:350`) doesn't either, hence the "Not available in DeploymentInfo" comment. The API adapters (`src/ai/backend/manager/api/adapters/deployment/adapter.py:2343`, `src/ai/backend/manager/api/rest/deployment/adapter.py:162`) just pass the empty list through.

This also contradicts the DTO's own contract: `ReplicaStateInfo.replica_ids` is documented as `"IDs of current replicas"` (`src/ai/backend/common/dto/manager/deployment/response.py:103`).

Confirmed on `26.4.6`, and re-confirmed on `main` at `763d07f49f` (2026-09-10).

## Why it matters

`replica_ids` is the only field on the REST deployment DTO that tells a client whether a `READY` deployment's replicas have actually come up. `READY` alone only means the rollout settled on the current revision — it does not assert any replica is serving. Consumers that gate on "are replicas provisioned?" by reading `len(replica_state.replica_ids)` therefore always see **zero** and conclude the deployment is not serving, even when it is healthy and routing traffic.

Concretely, this broke Backend.AI FastTrack: a model-serving pipeline task gated readiness on `len(replica_ids) >= desired_replica_count` and so was pinned in `PREPARING` forever (then failed at retry-timeout) for endpoints that were in fact serving. (Worked around there by deriving readiness from `search_routes()` instead — lablup/backend.ai-fasttrack#4781 / #4782 — but the DTO contract is still wrong for every other consumer.)

GraphQL clients are not affected in the same way: the `Deployment` type now exposes a `replicas` connection (`src/ai/backend/manager/api/gql/deployment/types/deployment.py:488`) that lists them directly. REST clients have no equivalent on the deployment DTO.

## Reproduction

1. Create a single-replica model deployment; let it reach `READY` with a running route.
2. `GET` the deployment (v2) or call the deployment service `get_deployment_by_id`.
3. Observe `status == READY`, a route with `status=running, traffic_status=active`, but `replica_state.replica_ids == []`.

## Proposed fix

Pick one of the following; either way, stop returning a field that silently lies.

**Option A — populate it.** Routes **are** the replicas (`RoutingRow.id` is a `ReplicaID`, and the route observer already does exactly this in `src/ai/backend/manager/sokovan/deployment/route/handlers/observer/health_check.py:79`: `replica_ids = [r.route_id for r in routes]`).

1. Load the deployment's routes in `EndpointRow.to_modern_deployment_info()` / `get_endpoint_info()` (or fetch them in the service before conversion).
2. Filter to active routes (`RouteStatus.active_route_statuses()` → `{PROVISIONING, RUNNING}`; decide whether `traffic_status == ACTIVE` should be required).
3. Set `replica_ids = [route.route_id for route in active_routes]` in both `_convert_deployment_info_to_data` and `_convert_deployment_info_to_legacy_data`.

Cost: a route load on every deployment read, including list endpoints.

**Option B — deprecate it.** Mark `replica_state.replica_ids` deprecated in both REST DTOs (v1 `response.py:103`, v2 `common/dto/manager/v2/deployment/types.py:139`), point clients to the replica/route listing (`search_replicas`, or the GraphQL `Deployment.replicas` connection), and drop the hard-coded converter lines once the deprecation window passes. No extra query per read, but REST clients need a second call to get replica state.

For Option A, decide explicitly whether `replica_ids` means "all current replicas" (PROVISIONING + RUNNING) or "serving replicas" (RUNNING only) and document it on `ReplicaStateInfo.replica_ids`, since clients gate on the count.

Contributor guide

Open the contributing guide

Research direction

Start with the two converters in src/ai/backend/manager/services/deployment/service.py, then trace how EndpointRow.to_modern_deployment_info() and DeploymentRepository.get_endpoint_info() build DeploymentInfo. Reproduce the empty replica_ids response with a READY deployment and inspect the route-loading behavior. Before coding, get a decision on populating active route IDs versus deprecating the field; done means both DTO paths and their documented contract consistently reflect that choice.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.