dssg / dssg/triage

Support evaluation subsets in Audition

Open
#519 0 comments 0 reactions 1 assignee Assigned to @tweddielin View on GitHub
Dominant language
Jupyter Notebook
Stars
201
Forks
62
PR merge metrics
No merged PRs in 30d

Description

Audition should accept a query against the predictions table (and a label for the query) to use (and create if necessary) evaluations for a _subset_ of the modeling cohort. Tables required:

- `test_results.subset_evaluations`: same as the evaluations table but with a subset name
- `test_results.entity_subset`: a linking table showing which entities are in associated with each subset at each date

Looking for comments @thcrock @tweddielin @saleiro @rayidghani

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.