Experiment tree interface tweaking
- Dominant language
- Python
- Stars
- 14
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
At the CV level of trial, the subset level of trial, and at the trial level, we should be able to:
1) Average runs/subsets/trials together. Average score, average ROC curve etc. This makes sense if and only if the runs are indistinguishable (i.e. KFold cross validation)
2) Take the metric for the best run/subset/trial. This makes sense if averaging things together is nonsense (i.e. sliding window cross validation.)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the experiment-tree interface and the code that computes or displays metrics at CV, subset, and trial levels. Review how KFold and sliding-window runs are represented, then define how averaging and best-run selection should appear, including score and ROC-curve behavior. Done means each level supports the appropriate aggregation without combining indistinguishable and distinguishable runs incorrectly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100