Medical-Event-Data-Standard / Medical-Event-Data-Standard/MEDS-DEV

MIMIC-IV `predicates.yaml` lab codes are `LAB//<itemid>//<unit>`, but the pinned ETL now emits `LAB//RESULT//<itemid>//<unit>`

Open Beginner friendly
#315 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
43
Forks
10
PR merge metrics
No merged PRs in 30d

Description

Summary

src/MEDS_DEV/datasets/MIMIC-IV/predicates.yaml references labevents lab codes as
LAB//<itemid>//<unit>, but the pinned ETL emits measured lab results as
LAB//RESULT//<itemid>//<unit> (with the value-less order event as
LAB//SPECIMEN_COLLECTED//<itemid>). On a freshly built MIMIC-IV MEDS dataset the labevents
predicates therefore match zero events.

This is an internal inconsistency on main: datasets/MIMIC-IV/requirements.txt pins
MIMIC-IV-MEDS==0.1.0 (the release that produces LAB//RESULT//), while predicates.yaml
still uses the older LAB//<itemid> scheme. Only the labevents itemids (5xxxx) are affected;
chartevents itemids (22xxxx) still emit LAB//<itemid>//<unit> and are fine.

Evidence

creatinine_1LAB//50912//mg/dL. In the built data:

LAB//50912//mg/dL               -> absent (0 events)
LAB//RESULT//50912//mg/dL       -> present  (the measured result)
LAB//SPECIMEN_COLLECTED//50912  -> present  (value-less order event)

Reproduce:

import glob, polars as pl
files = glob.glob('<mimic_meds_root>/data/**/*.parquet', recursive=True)
codes = set()
for f in files[:30]:
    codes |= set(pl.scan_parquet(f).select('code').unique().collect().to_series())
print('LAB//50912//mg/dL' in codes)          # False
print('LAB//RESULT//50912//mg/dL' in codes)  # True

Impact on the registered tasks (all supported_datasets: [MIMIC-IV])

Because creatinine = or(creatinine_1, creatinine_2) and both members are labevents:

  • abnormal_lab/blood_chemistry/elevated_creatininecreatinine and
    abnormally_high_creatinine both resolve to labevents-only → match nothing → the target
    window has: creatinine (1, None) is never satisfied → empty / degenerate cohort.
  • hyponatremia (sodium), metabolic_acidosis (bicarbonate), anemia (hemoglobin),
    leukocytosis (wbc), thrombocytopenia (platelets)
    — each or(...) keeps its surviving
    chartevents member, so the task still runs but silently drops the labevents
    contribution
    (biased cohort / shifted label prevalence).
  • hypotension (MAP), mortality/in_icu — MAP is all chartevents; mortality is structural →
    unaffected.

Proposed fix

Insert the RESULT// infix on the 8 labevents codes (each appears twice — a plain and an
abnormally_* variant); leave the chartevents (22xxxx) codes as-is. E.g.:

  creatinine_1:
    code: LAB//RESULT//50912//mg/dL
  creatinine_2:
    code: LAB//RESULT//52546//mg/dL
  abnormally_high_creatinine_1:
    code: LAB//RESULT//50912//mg/dL
    value_min: 1.3
    value_min_inclusive: False
    value_max: null
  # ...and likewise abnormally_high_creatinine_2, sodium_2/3, abnormally_low_sodium_2/3,
  # bicarbonate_2, abnormally_low_bicarbonate_2, hemoglobin_2, abnormally_low_hemoglobin_2,
  # wbc_2, abnormally_high_wbc_2, platelets_2, abnormally_low_platelets_2.

Full list of code strings to change (16 predicate entries, 8 distinct codes):
LAB//50912//mg/dL, LAB//52546//mg/dL, LAB//50983//mEq/L, LAB//52623//mEq/L,
LAB//50882//mEq/L, LAB//50811//g/dL, LAB//51300//K/uL, LAB//51265//K/uL
→ each prefixed with RESULT//.

Longer term: a check that catches predicate ↔ ETL drift

This class of bug (a predicate silently matching zero events after an ETL code-schema change)
is invisible today — the task still "runs," just on an empty or biased cohort. Some options to
catch it in CI:

  • Vocabulary check. Validate predicate codes against a reference code vocabulary for the
    pinned ETL version. (Note: metadata/codes.parquet is currently incomplete — it only lists
    codes that got a metadata-crosswalk join and omits chartevents vitals / structural / death
    codes — so this check should scan the data vocabulary, not codes.parquet.)
  • Couple the pin to the predicates. When requirements.txt bumps a dataset's ETL pin, require
    the predicate-validation job above to pass in the same PR, so the code schema and the predicates
    can't drift apart again.

Happy to send a PR for the predicate fix once the
direction is confirmed.

(generated with Claude, reviewed manually)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/MEDS_DEV/datasets/MIMIC-IV/predicates.yaml and compare its labevents codes with the pinned MIMIC-IV-MEDS==0.1.0 output described in datasets/MIMIC-IV/requirements.txt. Verify the listed 16 predicate entries use RESULT// for labevents while chartevents codes remain unchanged, then reproduce the code-vocabulary scan to confirm the predicates match measured results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.