dssg / dssg/pgdedupe

Store labeled examples in a table

Open
#63 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
43
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Labeled training example pairs should be stored in a table for selection and reuse. Data stored for examples should include:

- Source
- Source ids
- Label
- Label date
- Comment to store labeling rules applied by labeler

Storing examples like this allows them to by reused in the following ways:

- Select specific subsets of labeled pairs to build models from
- Store multiple labels for the same pair to determine how different labeling rules affect results
- Dynamically format labeled examples for models that use different set of input data by linking examples to source data tables

This will also entail a test that any model is only trained on one label per pair!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.