epam / epam/ai-dial-admin-evaluation-framework-backend
[Eval] Register EF as a DIAL Application to eliminate mid-run JWT expiry and enable file auto-sharing
- Dominant language
- Java
- Stars
- 3
- Forks
- 1
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 30
Description
### Name and Version
0.1.0
### What is the problem this feature will solve?
During long evaluation runs (minutes to hours), the Evaluation Framework (EF) captures the user's JWT at run-creation time and propagates it to asynchronous worker threads that invoke DIAL Core deployments. Two issues arise:
1. **JWT expiry mid-run** — user tokens expire while the run is still executing, causing `401 Unauthorized` failures on downstream deployment invocations and leaving runs in an inconsistent state.
2. **No file auto-sharing** — when evaluated deployments receive file attachments from the EF bucket (e.g. multi-modal test case inputs), they cannot access those files because EF is not a DIAL-registered application and DIAL Core's auto-sharing mechanism is not engaged.
### What is the feature you are proposing to solve the problem?
Register EF as a **DIAL Application** via DIAL Application Routes. DIAL Core would then issue a per-request key (PRK) that lives for the duration of the eval run connection, carries user identity and cost attribution through the full call tree, and enables DIAL Core's auto-sharing of EF-bucket files to evaluated deployments.
High-level behavior:
- DIAL Core triggers eval execution through an Application Route, passing a PRK to EF for the lifetime of that run.
- EF maps the PRK to the run and uses it for all outbound deployment invocations instead of the original user JWT, removing the expiry problem.
- The SSE connection between DIAL Core and EF is held open for the duration of the run to keep the PRK alive and to stream heartbeat/progress back to the caller.
- User identity is resolved from the PRK via DIAL Core's user-info endpoint rather than by parsing JWT claims directly.
- File attachments stored in the EF bucket become accessible to evaluated deployments automatically through DIAL Core's PRK-based auto-sharing.
- A feature flag gates the new behavior (default off) so the change can be rolled out gradually per environment, with the existing JWT-propagation path retained as a fallback.
No public REST API changes. No database schema changes. The internal endpoint used by DIAL Core is not exposed through the public ingress — it is reachable only via the DIAL Core route proxy.
### What alternatives have you considered?
- **JWT refresh-token propagation** — require the client to supply a long-lived refresh token and have EF refresh the access token mid-run. Rejected: increases security surface (EF would need to store refresh tokens) and does not solve the file auto-sharing problem.
- **Service-account / shared API key for outbound calls** — invoke deployments with a shared EF service key. Rejected: loses per-user identity and cost attribution, does not trigger DIAL Core auto-sharing, and breaks per-user quota enforcement.
- **Pre-signed URLs for EF-bucket files** — generate short-lived URLs and embed them in deployment payloads. Rejected: does not generalize to non-HTTP deployment types (e.g. MCP tools), duplicates auto-sharing logic that DIAL Core already implements, and introduces URL-expiry semantics that must be managed per run.
- **Keep JWT propagation, shorten run duration** — chunk long runs into sub-runs that fit inside token TTL. Rejected: adds user-visible complexity and still does not address file access.
Contributor guide
Assessment
This issue has not been assessed yet.