epam / epam/ai-dial-admin-evaluation-framework-backend
[Eval] Migrate to new /v1/deployments/{deployment_name} endpoint supplied by Core
- Dominant language
- Java
- Stars
- 3
- Forks
- 1
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 27
Description
### What is the problem this feature will solve?
**1. By-id deployment lookup is a 3-request workaround.**
`GET /api/v1/deployments/all/**` (`DeploymentService.getDeployment(String)`) resolves a deployment whose type the caller does not know by **probing three DIAL Core endpoints concurrently**:
```
GET /openai/models/{id}
GET /openai/applications/{id}
GET /openai/toolsets/{id}
```
and collapsing the outcomes via `DeploymentProbeCollapser` (two legs are expected to 404, one wins). This exists only because Core had no single-item, type-agnostic endpoint. It costs:
- 3× upstream traffic per lookup (the FE calls this on every suite summary page — see #179);
- a per-request virtual-thread executor + `TokenPropagationHelper` dance just to fan out;
- non-trivial collapse semantics (`DeploymentProbe`, `DeploymentProbeCollapser`, 404-vs-anomaly logging, mixed failures → unified 502) with their own unit + functional tests;
- inconsistent payloads: the by-id path returns the **per-type** shape, the list path (`GET /v1/deployments`) returns the **unified** `DeploymentData` shape.
DIAL Core has now shipped the single-item counterpart of the unified listing — [`GET /v1/deployments/{deployment_id}`](https://github.com/epam/ai-dial-core/pull/1922) (merged 2026-09-09). It returns the deployment in exactly the same `DeploymentData` shape the listing uses, resolves both config names (`gpt-4`) and resource-backed ids (`applications/{bucket}/{name}`), and answers `403` / `404` / `400` itself. The workaround is obsolete.
**2. We cannot tell which invocation API a deployment supports.**
The unified `DeploymentData` shape carries an `interfaces` array (`openaiChatCompletions`, `openaiResponses`, `openaiEmbeddings`, `anthropicMessages`, `mcp`, …). We already deserialize it into `DialCoreDeploymentDto.interfaces`, but:
- the per-type endpoints the by-id probe hits do **not** return `interfaces`, so the by-id response can never carry it;
- `DeploymentMapper` never maps `interfaces` onto `DeploymentInfoDto`, so the EF API exposes it **nowhere** — not even on the list endpoint.
This is the blocker for **#20 (OpenAI Responses API)** and **#168 (Anthropic Messages API)**: both need the FE (and suite validation) to know *definitively* whether a chosen deployment supports `/openai/v1/responses` or `/anthropic/v1/messages` before offering that endpoint type. Today the only signal is a heuristic on `object` type / `deploymentRef.type`, which cannot distinguish a chat-completions-only model from a Responses-capable one. It also feeds #188 (deployment-id consistency for shared-route bodies), which needs the interface to decide whether a request template goes through a deployment-scoped or a shared route.
### What is the feature you are proposing to solve the problem?
**A. Replace the 3-probe lookup with the new Core endpoint**
- `DialCoreClient`: add `getDeployment(String id)` → `GET /v1/deployments/{id}` returning `DialCoreDeploymentDto` (polymorphic on `object`, same as the list). Path-encode the id once; ids may contain `/` (`applications/{bucket}/{name}`), which must not be double-encoded (see `WildcardPathResolver` / slash-path conventions).
- `DeploymentService.getDeployment(String)`: single call to the new client method, mapped with the same mapper used for list entries so **by-id and list payloads are identical**. Pass Core's `403` / `404` / `400` through as today's `DialCoreClientException`.
- Delete `DeploymentProbe`, `DeploymentProbeCollapser`, `probeAsync`/`awaitProbe`/`logProbeFailure`, and the executor + token propagation in `getDeployment(String)`. Remove `DeploymentProbeCollapserTest`; rewrite `DeploymentServiceTokenPropagationTest` and the by-id cases in `DeploymentFunctionalTests` against the new upstream call (WireMock: one stub instead of three).
- Keep the by-type path (`GET /api/v1/deployments/{type}/**` → `/openai/{models|applications|toolsets}/{id}`) untouched: it remains the only source of per-type-only fields (`capabilities`, `limits`, `pricing`, `applicationProperties`, `routes`, `allowedTools`) and of `SchemaRouteExtractor` route resolution.
**B. Expose `interfaces` on the EF API**
- Add `interfaces: List` to `DeploymentInfoDto` (or the relevant per-type subtypes) and map it in `DeploymentMapper.toDeploymentInfoShortDto` and the new by-id mapping. Serialize with Core's wire values (`openaiChatCompletions`, …).
- Populated on `GET /api/v1/deployments` and `GET /api/v1/deployments/all/**`. On the by-type path it stays `null`/absent (Core does not return it there) — document this.
- Update OpenAPI `@Schema` examples + `src/main/resources/openapi/examples/` for both endpoints.
**C. Housekeeping**
- Update `openspec/specs/dial-core-client/spec.md` (by-id scenarios, payload parity, `interfaces`) and the `DeploymentController` OpenAPI descriptions that still describe the "probe all three" behaviour. No new configuration property expected.
- Bump the minimum supported DIAL Core version in README / compatibility notes to the first release containing ai-dial-core#1922.
**Acceptance criteria**
- [ ] `GET /api/v1/deployments/all/{id}` issues exactly **one** upstream request (`GET /v1/deployments/{id}`) — asserted in functional tests via WireMock request counting.
- [ ] Response body of `GET /api/v1/deployments/all/{id}` equals the matching entry of `GET /api/v1/deployments` for a model, an application (incl. `applications/{bucket}/{name}` id), and a toolset.
- [ ] Core `404` → EF `404`; Core `403` → EF `403`; Core `400` (id parses as a non-deployment resource URL) → EF `400`; transport failure → EF `502`.
- [ ] `interfaces` is present and correct on list + by-id responses; absent on by-type responses.
- [ ] `DeploymentProbe*` classes and tests are gone; `LayeredArchitectureTest`, `LoggingConventionTest`, checkstyle, spotless pass.
- [ ] OpenAPI examples updated and asserted via `/v3/api-docs` test.
**Out of scope**
- Consuming `interfaces` in suite validation / FE endpoint-type selection — that is the body of #20, #168, #188. This issue only makes the data available.
- Removing the per-type by-id endpoints or the `/openai/*` client methods.
### What alternatives have you considered?
- **Keep the 3-probe lookup and just add `interfaces` via a 4th call to `GET /v1/deployments` (list) and filter client-side** — rejected: makes the hot path *worse* (4 requests, one of them the full listing), which is the exact inefficiency #179 flagged.
- **Fetch `GET /v1/deployments/{id}` first for `interfaces`, then the per-type endpoint for the rich fields, in one by-id call** — rejected for the `all/**` path: doubles latency for the FE summary page and re-introduces two payload shapes. Callers that need per-type-only fields already have `GET /api/v1/deployments/{type}/**`.
- **Infer supported APIs from `object` type / `features` instead of `interfaces`** — rejected: not definitive (a `model` may or may not expose `openaiResponses` / `anthropicMessages`), and it is precisely the ambiguity Core introduced `interfaces` to remove.
- **Feature-flag the new lookup behind a property to support older Core versions** — rejected: doubles the code paths and tests for a transitional period; a minimum-Core-version bump is cheaper. Can be revisited if deployments against pre-#1922 Core must be supported.
### Related
- Upstream: https://github.com/epam/ai-dial-core/pull/1922
- Closes #179 (spike answered: yes, `GET /v1/deployments/{id}`)
- Unblocks #20, #168; prerequisite for #188
Contributor guide
Assessment
This issue has not been assessed yet.