dotnet / dotnet/machinelearning
RegressionEvaluator return 0 for MSE && RMSE if all scores are NaN
Open
area-Core
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
It seems that RegressionEvaluator filters out all `NaN` scores when calculating metrics. So if a predictor produces `NaN` only for the entire evaluation dataset, the RMSE and MSE from RegressionEvaluator would be 0, which is ironically the best result it can get.
While it may need further discussion, I wonder if it's necessary to _FILTER_ out the `NaN` value when calculating metrics, as `NaN` in prediction result usually indicates a model is not well-trained, and thus should be reflected in evaluation metrics.
Contributor guide
Assessment
This issue has not been assessed yet.