Support evaluation subsets in Audition
Open
- Dominant language
- Jupyter Notebook
- Stars
- 201
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
Audition should accept a query against the predictions table (and a label for the query) to use (and create if necessary) evaluations for a _subset_ of the modeling cohort. Tables required:
- `test_results.subset_evaluations`: same as the evaluations table but with a subset name
- `test_results.entity_subset`: a linking table showing which entities are in associated with each subset at each date
Looking for comments @thcrock @tweddielin @saleiro @rayidghani
Contributor guide
Assessment
This issue has not been assessed yet.