facebookresearch / facebookresearch/fairo

Manage versioned NLU models and datasets in S3

Open
#308 2 comments 0 reactions 2 assignees Claimed by @anuragprat1k View on GitHub
P2
Dominant language
Jupyter Notebook
Stars
929
Forks
123
PR merge metrics
No merged PRs in 30d

Description

## Type of Issue

Select the type of issue:
- [ ] Bug report (to report a bug)
- [x] Feature request (to request an additional feature)
- [ ] Tracker (I am just using this as a tracker)
- [ ] Refactor request
- [x] Documentation Ask

## Description

We should update the datasets and models distribution protocol with how to manage versioned binaries.

## Current Behavior

All past models and dataset versions are stored in s3 with filename `_folder_.tar.gz`.

Some concerns
- Do we want to keep all past versions? I actually think this is fine, as in the benefits of reproducibility outweigh the storage costs. S3 is preferred for longterm storage over FAIR checkpoints.
- Hashes are non descriptive. If we want to keep older versions around, we should at least tag them or maintain READMEs in the releases. Right now Turk jobs with dashboard use different versions than master. So we at least want two tags `master` and `stable` going forward.
- People have suggested including multiple models in the models release itself, personally against this idea because old models need to be used with a specific dataset release, so this becomes hard to track. Also want to keep the binaries small.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.