facebookresearch / facebookresearch/fairo
Manage versioned NLU models and datasets in S3
- Dominant language
- Jupyter Notebook
- Stars
- 929
- Forks
- 123
- PR merge metrics
- No merged PRs in 30d
Description
## Type of Issue
Select the type of issue:
- [ ] Bug report (to report a bug)
- [x] Feature request (to request an additional feature)
- [ ] Tracker (I am just using this as a tracker)
- [ ] Refactor request
- [x] Documentation Ask
## Description
We should update the datasets and models distribution protocol with how to manage versioned binaries.
## Current Behavior
All past models and dataset versions are stored in s3 with filename `_folder_.tar.gz`.
Some concerns
- Do we want to keep all past versions? I actually think this is fine, as in the benefits of reproducibility outweigh the storage costs. S3 is preferred for longterm storage over FAIR checkpoints.
- Hashes are non descriptive. If we want to keep older versions around, we should at least tag them or maintain READMEs in the releases. Right now Turk jobs with dashboard use different versions than master. So we at least want two tags `master` and `stable` going forward.
- People have suggested including multiple models in the models release itself, personally against this idea because old models need to be used with a specific dataset release, so this becomes hard to track. Also want to keep the binaries small.
Contributor guide
Assessment
This issue has not been assessed yet.