Would an automatically generated dependency graph for MIMIC-IV concepts be useful?
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 3.4k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for maintaining the MIMIC code repository.
While looking through the MIMIC-IV concept build process, I noticed that the execution order of derived concepts is currently partly encoded manually. For example, make_concepts.sh handles several dependency-sensitive concepts separately, such as first_day_sofa, kdigo_stages, sepsis3, and the vasoactive-agent related concepts, so that their prerequisites are created first.
I also noticed that SQLGlot is already used in the repository for parsing and transpiling concept SQL across different dialects. This made me wonder whether it could also be used to infer dependencies between derived concepts automatically.
For example, references to tables under mimiciv_derived.* could be extracted from each concept SQL file to identify partial dependency relationships such as:
urine_output → kdigo_uo
kdigo_uo → kdigo_stages
sofa → sepsis3
suspicion_of_infection → sepsis3
These relationships could then be used to construct a directed dependency graph.
Once such a graph is available, it might be useful for:
- automatically determining a valid concept build order through topological sorting;
- detecting missing or circular dependencies;
- reducing the need to maintain dependency-sensitive execution order manually.
It could potentially also provide a machine-readable description of the dependencies among derived concepts.
Since SQLGlot is already part of the current workflow, I was wondering whether this kind of dependency analysis would be useful for the project, or whether there are aspects of the current concept-generation process that would make automatic dependency inference unreliable.
Thanks for your thoughts.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with make_concepts.sh and the derived concept SQL files, then review how SQLGlot is used for parsing and transpilation. Determine whether references under mimiciv_derived.* can reliably support dependency extraction, including missing or circular dependencies. Done would require an agreed scope for automatic ordering or graph generation, rather than only confirming that it may be useful.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell, sql
- Domain
- build-system, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100