Medical-Event-Data-Standard / Medical-Event-Data-Standard/MEDS-DEV

Add concrete schemas for result blobs and entity manifests

Open
#290 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Code Cleanliness/Tech Debt priority:medium Results storage and tracking Website / Branding
Dominant language
Python
Stars
43
Forks
10
PR merge metrics
No merged PRs in 30d

Description

Follow-up to PR #283 (https://github.com/Medical-Event-Data-Standard/MEDS-DEV/pull/283#discussion_r3219845115).

Today the result.json blobs on _results and the entity manifests on _web (datasets.json, tasks.json, models.json) are both implicitly shaped — readers (notably the website's TypeScript types in src/lib/MEDS-DEV/types.ts) have to encode the schema by hand, and producers (MEDS_DEV.results.Result, MEDS_DEV.web.collate_entities) encode it by hand on the other side.

That's three places that drift independently.

Proposal

Add concrete schemas for the two file types so producers, consumers, and validators share a single source of truth.

Options for the schema mechanism:

  1. JSON Schema (vanilla) — broadest interoperability; both jsonschema (Python) and Ajv (JS) read it; can ship next to the JSON files for online validation. Loses Python type integration.
  2. flexible_schema — sibling project; designed for this kind of cross-language schema contract; can emit JSON Schema. (Author preference per the review thread.)
  3. Pydantic models — Python-native, can emit JSON Schema, but ties the consumers more tightly to a Python lib. Less attractive than flexible_schema if the latter does what we need.

Coverage:

  • Result blob schema. The shape is already encoded in MEDS_DEV.results.Result (dataclass + _sanitize_nan_inf etc.); much of this may belong upstream in meds-evaluation since the inner result dict comes from there. Decide where the schema's home is.
  • Aggregated results schema. {<issue_number>: <result-blob>} is just a dict-of-result-blobs; trivially derived from the per-blob schema.
  • Entity manifests schema (datasets / tasks / models). Currently mirrored by hand in src/lib/MEDS-DEV/types.ts on the website side.
  • Validation hooks in the producers — meds-dev-validate-result, meds-dev-pack-result, meds-dev-collate-entities all run schema validation before writing.
  • Website-side adoption of the published schema (separate issue in the website repo).

Coupling with #289

#289 tracks unification of entity-walking code paths and the version-aware regen pivot. The entity schema work probably wants to be done together with — or after — the unification half of #289, because the choice of "registries as source of truth" vs "separate walker" affects where the producer-side schema lives.

The result-blob schema is largely independent of #289.

Acceptance criteria

  • Pick a schema mechanism (JSON Schema vs flexible_schema vs Pydantic).
  • Result-blob schema in place; decide home (this repo vs meds-evaluation).
  • Entity-manifest schema in place; producer validates before writing.
  • Schemas published to a stable location the website can consume (probably on _web next to the data files they describe).
  • Website-side issue filed for adopting the schemas as the source of truth for types.ts.

Related

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with MEDS_DEV.results.Result, MEDS_DEV.web.collate_entities, and the producer commands named in the issue; compare their output with src/lib/MEDS-DEV/types.ts. Review the relationship to #289 before choosing the schema boundary and mechanism. Done means the result and entity-manifest schemas, producer validation hooks, stable publication location, and website follow-up are clearly assigned or implemented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
backend-api-design, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.