EducationalTestingService / EducationalTestingService/rsmtool
More prominent warning for classifiers that do not support expected probabilities
Open
enhancement
help wanted
- Dominant language
- Python
- Stars
- 71
- Forks
- 21
- PR merge metrics
- No merged PRs in 30d
Description
For classifiers that do not support expected probabilities, we currently rely on SKLL to raise a warning and proceed generating integer scores. The final report still says "Predictions analyzed in this report are expected scores".
This is easy to miss when running multiple experiments.
We might consider either raising an error during model training or at least changing the wording in the report.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.