dotnet / dotnet/machinelearning

RegressionEvaluator return 0 for MSE && RMSE if all scores are NaN

Open
#6,361 3 comments 0 reactions 0 assignees View on GitHub
area-Core
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

It seems that RegressionEvaluator filters out all `NaN` scores when calculating metrics. So if a predictor produces `NaN` only for the entire evaluation dataset, the RMSE and MSE from RegressionEvaluator would be 0, which is ironically the best result it can get.

While it may need further discussion, I wonder if it's necessary to _FILTER_ out the `NaN` value when calculating metrics, as `NaN` in prediction result usually indicates a model is not well-trained, and thus should be reflected in evaluation metrics.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.