EducationalTestingService / EducationalTestingService/rsmtool

More prominent warning for classifiers that do not support expected probabilities

Open
#196 0 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
Python
Stars
71
Forks
21
PR merge metrics
No merged PRs in 30d

Description

For classifiers that do not support expected probabilities, we currently rely on SKLL to raise a warning and proceed generating integer scores. The final report still says "Predictions analyzed in this report are expected scores".
This is easy to miss when running multiple experiments.

We might consider either raising an error during model training or at least changing the wording in the report.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.