deepchecks / deepchecks/deepchecks

[FEAT][ModelEvaluation] improve simple model comparison

Open
#1,572 1 comment 0 reactions 0 assignees View on GitHub
feature
Dominant language
Python
Stars
4.1k
Forks
303
PR merge metrics
No merged PRs in 30d

Description

There several ways in which this check could be improved:
1. Train and compare to several simple models options instead of just one
2. Added option of true random - randomly select a label from a uniform distribution of all labels (currently it is from true label distribution)
3. Think of more options for simple models
4. Add additional performance display options (maybe plot error graphs based on an interesting feature)
5. Select metrics / display based on task of model

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.