epam / epam/ai-dial-admin-evaluation-framework-backend

[Eval] Support invoking deployments via the OpenAI Responses API alongside /chat/completions

Open
#20 1 comment 0 reactions 1 assignee Claimed by @buhaiovos View on GitHub
enhancement FE Required P4 - Low
Dominant language
Java
Stars
3
Forks
1
Avg merge
1d 1h
Merged PRs (30d)
27

Description

### Name and Version

0.1.0

### What is the problem this feature will solve?

The Evaluation Framework (EF) currently invokes DIAL deployments primarily through the OpenAI-compatible `/v1/chat/completions` contract. OpenAI has standardized a newer **Responses API** (`/v1/responses`), and DIAL Core is gaining first-class support for it (see [ai-dial-core#1420 (https://github.com/epam/ai-dial-core/pull/1420)). Two problems follow from EF not yet supporting it as a peer invocation target:

1. **Coverage gap** — deployments that natively expose only the Responses API (or whose new capabilities are surfaced only there, e.g. richer structured output, native tool calls, multi-modal output items) cannot be evaluated by EF. Users must either fall back to a chat/completions-shaped wrapper or skip evaluation entirely.
2. **Drift from upstream** — as model providers and DIAL Core continue to invest in the Responses API as the forward-looking standard, EF risks lagging behind and forcing users to author brittle, hand-crafted generic-HTTP test suites for what should be a typed, first-class endpoint type.

### What is the feature you are proposing to solve the problem?

Add the **Responses API** as a first-class deployment-invocation target in EF, configured and used in the same way as the existing chat/completions target — same authoring experience for test suites, same request templating, same response-column extraction, same streaming and timeout semantics.

High-level behavior (v1 scope, stateless single-turn only):

- When defining a test suite, users can choose *Responses API* as the deployment endpoint type alongside the existing chat/completions option (for deployments which supports responses api).
- Request templates produce a Responses-API-shaped payload (input items, instructions, model parameters), with the same placeholder/binding mechanism used today for chat/completions.
- EF invokes the deployment through DIAL Core's Responses API route, propagating user identity and authentication consistently with other deployment calls.
- Streaming is supported on parity with chat/completions: incremental output items are consumed during a run, with progress surfaced through the existing run/SSE channels.
- Response payloads are normalized into the existing test-case-result model so that response-column extraction (JSONata, schema-driven) works identically regardless of endpoint type.
- Timeouts, retries, error mapping, and cost/usage capture behave the same as for chat/completions.

**Explicit non-goals for v1:**

- **Stateful features** of the Responses API — `previous_response_id`, server-side conversation persistence, stored/resumable responses — are out of scope. Only stateless single-turn invocation is supported in the first cut. Stateful evaluation flows can be designed as a follow-up once the stateless path is stable.
- No changes to EF's own public API shape; only a new value for the deployment endpoint type.
- No database schema changes beyond what is needed to record the chosen endpoint type per test suite.

### What alternatives have you considered?

- **Wrap Responses API behind a chat/completions adapter** — translate Responses-shaped payloads/responses into the existing chat/completions path. Rejected: loses Responses-specific capabilities (typed output items, native structured output, richer tool semantics), adds a translation layer that lags upstream changes, and gives users a misleading endpoint type in the UI.
- **Wait until the Responses API fully replaces chat/completions upstream** — defer until there is a single standard. Rejected: the two endpoints will coexist for an extended period; users need to evaluate Responses-only deployments today, and waiting accumulates a backlog of unsupported test suites.
- **Ship stateful and stateless support together** — include `previous_response_id` and stored conversations in the first cut. Rejected: stateful evaluation introduces independent design questions (run reproducibility, snapshotting of server-side state, cleanup/retention) that are best handled in a follow-up once the stateless path is in production.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.