Store labeled examples in a table
- Dominant language
- Jupyter Notebook
- Stars
- 43
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
Labeled training example pairs should be stored in a table for selection and reuse. Data stored for examples should include:
- Source
- Source ids
- Label
- Label date
- Comment to store labeling rules applied by labeler
Storing examples like this allows them to by reused in the following ways:
- Select specific subsets of labeled pairs to build models from
- Store multiple labels for the same pair to determine how different labeling rules affect results
- Dynamically format labeled examples for models that use different set of input data by linking examples to source data tables
This will also entail a test that any model is only trained on one label per pair!
Contributor guide
Assessment
This issue has not been assessed yet.