awslabs / awslabs/keys_values

Allow for dataset to be loaded from S3 bucket. Also train/valid split

Open
#147 1 comment 0 reactions 1 assignee Claimed by @mseeger View on GitHub
enhancement
Dominant language
Python
Stars
18
Forks
8
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
The Helmet datasets are created from scripts, using random choices. Also, train/valid splits of dev sets are random. This introduces variance in results when training/eval is done on different instances, or between ours and baseline.

**Describe the solution you'd like**
Ideally, we just create the datasets and train/valid splits on S3 and load them from there, so they are always the same.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.