demml / demml/opsml

[eval] EvalScenario schema + EvalRegistryRecord data model

Open
#382 0 comments 0 reactions 0 assignees View on GitHub
Backend enhancement
Dominant language
Rust
Stars
40
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Part of #380

## What

Define the core data model for the evaluation framework.

- `EvalScenario`: scenario definition struct — inputs, expected outputs, eval function reference, metadata, scenario hash
- `EvalRegistryRecord`: versioned registry entry linked to a `ServiceCard` (space/name/version/uid) and a Scouter record id
- DB migration for eval registry records table
- Scouter payload contract: what shape does Scouter bifrost expect for eval scenario registration

Contract types belong in `opsml-types`. DB schema goes through the standard migration path in `opsml-sql`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.