dotnet / dotnet/machinelearning
[ImageClassification Transfer Learning] Need simplified API for obtaining the List of predicted labels with their related scoring
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
In the current implementation of ImageClassification Transfer Learning (but this also happens when scoring regular TensorFlow models) the list of predicted labels (usually categorical data of type string/text) with their related scoring (probability) is not straightforward to get.
We need a very simple and straightforward way of getting the best predicted values (such as categorical data) with their related probability.
Currently, in a typical ImagePrediction class you might have (such as in the sample link):
https://github.com/dotnet/machinelearning/blob/bbb6b1560dbd67640cf9c4b41640e26769ee664d/docs/samples/Microsoft.ML.Samples/Dynamic/ImageClassification/InceptionV3TransferLearning.cs#L101
```
public class ImagePrediction
{
[ColumnName("Score")]
public float[] Score;
[ColumnName("PredictedLabel")]
public Int64 PredictedLabel;
}
```
1. First of all, the predicted label you get is a number (Int64) which is an **index**, not the value itself such as a categorical value. How is the user supposed to easily find out the text/categorical predicted label? - Sure, we might be able to find out through the schema API, but that is not straightforward.
2. With the `PredictedLabel` field we're only getting **one** predicted label (so far, its index), but the user would probably like to get an array with the "best" or even "all" predicted labels plus their correlated score/probability.
3. The float[] array `Score` is precisely getting the probabilities for all the labels. But, how is the user supposed to easily be able to correlate those scores with their related labels (categorical values, for instance)?
**ACTIONS MOVING FORWARD:**
1. As initial step, the sample code above (or new sample) should show on the console output the list of all labels in their original form (categorical/text), or at least the best three predicted labels and their related score/probability. This should be done in the sample with the current API capabilities even if it is not straightforward to code for a user.
2. Moving forward, since our goal is to create a simplified and high-level API for ImageClassification Transfer Learning, we should provide a way where such information is straightforward and **directly provided by the "Prediction class"**.
Contributor guide
Assessment
This issue has not been assessed yet.