MIT-LCP / MIT-LCP/mimic-code

Would an automatically generated dependency graph for MIMIC-IV concepts be useful?

Open
#2,158 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
3.4k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for maintaining the MIMIC code repository.

While looking through the MIMIC-IV concept build process, I noticed that the execution order of derived concepts is currently partly encoded manually. For example, make_concepts.sh handles several dependency-sensitive concepts separately, such as first_day_sofa, kdigo_stages, sepsis3, and the vasoactive-agent related concepts, so that their prerequisites are created first.

I also noticed that SQLGlot is already used in the repository for parsing and transpiling concept SQL across different dialects. This made me wonder whether it could also be used to infer dependencies between derived concepts automatically.

For example, references to tables under mimiciv_derived.* could be extracted from each concept SQL file to identify partial dependency relationships such as:

urine_output → kdigo_uo
kdigo_uo → kdigo_stages

sofa → sepsis3
suspicion_of_infection → sepsis3

These relationships could then be used to construct a directed dependency graph.

Once such a graph is available, it might be useful for:

  • automatically determining a valid concept build order through topological sorting;
  • detecting missing or circular dependencies;
  • reducing the need to maintain dependency-sensitive execution order manually.

It could potentially also provide a machine-readable description of the dependencies among derived concepts.

Since SQLGlot is already part of the current workflow, I was wondering whether this kind of dependency analysis would be useful for the project, or whether there are aspects of the current concept-generation process that would make automatic dependency inference unreliable.

Thanks for your thoughts.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with make_concepts.sh and the derived concept SQL files, then review how SQLGlot is used for parsing and transpilation. Determine whether references under mimiciv_derived.* can reliably support dependency extraction, including missing or circular dependencies. Done would require an agreed scope for automatic ordering or graph generation, rather than only confirming that it may be useful.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell, sql
Domain
build-system, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.