Medical-Event-Data-Standard / Medical-Event-Data-Standard/MEDS-DEV
We should track "hashes" or "signatures" of given datasets, task label dataframes, and model artifacts with experimental results
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
This will allow us to ensure we avoid any unintentional mis-attribution of task, model, dataset, results to differing versions, etc.
See https://github.com/Medical-Event-Data-Standard/meds/issues/53 for tracking collecting hashes on MEDS dataset objects.
Storing model artifact hashes is both easier and trickier -- simple checksums of saved file contents is apparently insufficient, and I don't know if a hashlib conversion of the model state dict would work either: https://discuss.pytorch.org/t/why-does-the-hash-value-of-models-change-everytime-i-save-it/24249/4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing MEDS issue 53 and the linked PyTorch discussion about model hashes. Define how hashes or signatures should cover datasets, task-label dataframes, model artifacts, and experimental results, then document what reproducible identifiers and validation behavior would count as done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100