Allow for dataset to be loaded from S3 bucket. Also train/valid split
Open
enhancement
- Dominant language
- Python
- Stars
- 18
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
The Helmet datasets are created from scripts, using random choices. Also, train/valid splits of dev sets are random. This introduces variance in results when training/eval is done on different instances, or between ours and baseline.
**Describe the solution you'd like**
Ideally, we just create the datasets and train/valid splits on S3 and load them from there, so they are always the same.
Contributor guide
Assessment
This issue has not been assessed yet.