Medical-Event-Data-Standard / Medical-Event-Data-Standard/MEDS-DEV

Add larger MEDS-Tab variants alongside meds_tab/tiny

Open
#303 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement MEDS-Tab Models
Dominant language
Python
Stars
43
Forks
10
PR merge metrics
No merged PRs in 30d

Description

The only MEDS-Tab variant currently registered is meds_tab/tiny — explicitly described as "a tiny, low-cost XGBoost baseline." Add at least one (ideally two) larger variants so MEDS-Tab can be evaluated as a competitive baseline rather than just a smoke-test.

What this needs

New sibling dirs under src/MEDS_DEV/models/meds_tab/, e.g.:

  • meds_tab/standard/ — moderate-capacity sweep (more codes, more window sizes, more trials).
  • meds_tab/large/ — wider sweep / more aggregation types, intended as the "best-effort" tabular baseline.

Naming is up to @Oufattole / @teyaberg — the point is capacity variants, not new framework features.

Knobs to scale up

Pulling from meds_tab/tiny/model.yaml, the obvious dimensions are:

  • tabularization.aggs — currently [code/count, value/sum]. Add value/min, value/max, value/mean, value/std, etc.
  • tabularization.window_sizes — currently [7d, 30d]. Add shorter (1h, 6h, 24h) and longer (90d, 365d) windows.
  • tabularization.max_included_codes — currently 100. Bump to 1000 / 10000 / unbounded.
  • tabularization.min_code_inclusion_count — currently null. Could constrain for the larger sweep.
  • hydra.sweeper.n_trials — currently 20. Bump for the larger variants.
  • hydra.sweeper.n_jobs — currently 1. Parallelize where appropriate.

Open questions

  • Capacity nomenclature: tiny/small/base/large vs tiny/standard/large vs something else?
  • Where does the gradient from "tiny" to "large" sit on real benchmark wall-clock — worth pinning at least one variant's hyperparams to what was used in the MEDS-Tab paper?

Pinging @Oufattole, @teyaberg — best to take the lead on what hyperparams make sense.

Refs

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/MEDS_DEV/models/meds_tab/tiny/model.yaml and inspect how sibling model directories are registered. Compare the tiny settings with the MEDS-Tab paper and resolve the capacity naming and hyperparameters with the mentioned maintainers. Done means at least one larger sibling variant is configured and available for benchmark evaluation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, yaml
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.