Medical-Event-Data-Standard / Medical-Event-Data-Standard/MEDS-DEV
Add workflow to auto-regenerate _web entity manifests on source changes
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
Background
Even once the collation script is packaged (#280), there is currently no workflow that regenerates _web/entities/{datasets,tasks,models}.json when a dataset / task / model changes on main. The website happily fetches stale data forever.
Today the entity files on _web were last updated 2025-05-20 (Update prelim entity files.) and are visibly stale (missing AUMCdb, missing several abnormal_lab/cbc and vital/hypotension tasks, embedded MIMIC-IV config still uses the obsolete MEDS_cohort_dir= parameter).
What needs to be done
Add a workflow .github/workflows/regenerate_entities.yaml that:
- Triggers on push to
mainwhen files undersrc/MEDS_DEV/{datasets,tasks,models}/**change. Also exposesworkflow_dispatchfor manual runs. - Checks out
main(for source) and_web(in a separate path, for output). - Installs MEDS-DEV from the local checkout so it picks up the latest collation script.
- Runs
meds-dev-collate-entities --repo_dir . --output_dir <web>/entities --do_overwrite(CLI shape per #280). - Commits and pushes to
_webif anything changed. Skip the commit if no diff.
Open questions
- Should the workflow run on every push to
main, or only on PR merges that touched the relevant paths? (Path filter onpaths:answers this naturally.) - For the push back to
_web: sameGITHUB_TOKENtriggering issue described in #238 applies if any downstream automation is ever wired up to react to_webchanges. For now there's no downstream — the website fetches at request time — so defaultGITHUB_TOKENis fine.
Acceptance criteria
- Editing any
dataset.yaml/task.yaml/model.yaml(or related files) in a merged PR results in_web/entities/*.jsonbeing regenerated within minutes. - Stale entries on
_web(e.g. missing AUMCdb) appear after the workflow runs once. - Workflow is idempotent (running it twice yields no new commits).
Related
- Depends on #280 (CLI entry point)
- Resolves the automation half of #186, #187, #188
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .github/workflows and the collation CLI from issue #280, then review the source paths under src/MEDS_DEV/{datasets,tasks,models}/ and the _web checkout layout. Add and manually run regenerate_entities.yaml, verifying that entity JSON files are regenerated, changes are committed only when present, and a second run is idempotent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100