Validate model groups all have results for same/full set of end times
Open
audition
- Dominant language
- Jupyter Notebook
- Stars
- 201
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
_From @shaycrk on August 15, 2017 20:17_
If the model groups we're considering haven't all been tested on the same set of test sets, we might not be making fair comparisons for model selection, so we should check the input to ensure the a uniform set of test sets are used throughout.
_Copied from original issue: dssg/audition#18_
Contributor guide
Research direction
No files or tests are named. Locate the input handling for model groups and the existing validation tests, then trace how test sets or end times are represented. Done means inputs with differing test sets are detected so model comparisons use a uniform set throughout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100