dotnet / dotnet/machinelearning

FastForestOva has its prediction value in Score[1] instead of Score[0]

Open
#6,037 5 comments 1 reaction 0 assignees View on GitHub
area-Trees
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**
- OS & Version: Windows 11
- ML.NET Version: ML.NET v1.7
- .NET Version: .NET 6.0
- ML.NET Model Builder: ML.NET Model Builder 2022 v16.9.2.2205603

**Describe the bug**
Usually the probabilities of the trained models are stored in the Score values of the `ModelOutput` class (generated by ML.NET Model Builder). The first item in that array of floats is the prediction value when we use the `FastTreeOva` or `LightGbmMulti` algorithms, but interestingly it is the second item in that array if we use `FastForestOva`.
I'm not 100% sure that it is a bug, but I would expect consistency in this regard among the different algorithms.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to Solution Explorer
2. Right click on the project, add Machine Learning Model
3. Start training your model, and stop at the different times to have different algorithms as best
4. Test the generated classes by running them with your input data and show the ModelOutput.Score array
5. Observe the different values and their meaning, especially for the `FastForestOva` algorithm

**Expected behavior**
I would expect all the algorithms to return their prediction values in the same order.

**Screenshots, Code, Sample Projects**
Accuracy is calculated based on `OutputModel.Score[0]` values and `IsAttached` uses `LightGbmMulti`, `IsNotAttached` uses `FastForestOva`. They both had 99+% accuracy during training.
![image](https://user-images.githubusercontent.com/910321/148792403-a89a6af2-e3c8-4534-b3ce-3b80c05dd83e.png)

**Additional context**
I could test `FastTreeOva`, `LightGbmMulti` and `FastForestOva`: the first two had their prediction value in Score[0], and `FastForestOva` had its in Score[1].

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.