dotnet / dotnet/machinelearning

Saved model returning a different prediction when using AddPredictionEnginePool

Open
#6,913 0 comments 0 reactions 1 assignee Claimed by @rosebyte View on GitHub
area-Integration
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**
- OS & Version: Windows 10
- ML.NET Version: 3.0.0
- .NET Version: 7.0

**Describe the bug**
I'm trying to build something with an API endpoint exposed. While prototyping, I'm essentially following the documentation here: https://learn.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/serve-model-web-api-ml-net

I would expect the following two blocks to give the same result:

(In a console app)
```
var mlContext = new MLContext();

SentimentIssue sampleStatement = new SentimentIssue { Text = "I love this movie!" };

string ModelPath = "path/to/SentimentModel.zip";

ITransformer loadedModel = mlContext.Model.Load(ModelPath, out var modelInputSchema);
var predEngine = mlContext.Model.CreatePredictionEngine(loadedModel);
var resultprediction = predEngine.Predict(sampleStatement);

Console.WriteLine($"=============== Single Prediction ===============");
Console.WriteLine($"Text: {sampleStatement.Text} | Prediction: {(Convert.ToBoolean(resultprediction.Prediction) ? "Positive" : "Negative")} sentiment | Probability of being positive: {resultprediction.Probability} ");
```
Actual output:
![image](https://github.com/dotnet/machinelearning/assets/105062729/58aa22e5-f368-4231-9769-1ac6c466433b)

In an API, in the Program.cs file:
```
builder.Services.AddPredictionEnginePool().FromFile(modelName: "SentimentAnalysisModel", filePath: "path/to/SentimentModel.zip", watchForChanges: true);

// ...

var predictionHandler =
(PredictionEnginePool predictionEnginePool, SentimentAnalysisModelInput input) =>
{
return predictionEnginePool.Predict(modelName: "SentimentAnalysisModel", input);
};

app.MapPost("/predict", predictionHandler);

app.Run();

```

The output (for example, using Postman with the same input)
![image](https://github.com/dotnet/machinelearning/assets/105062729/3881d198-6493-40a1-9f37-77c2cf29eed3)

**Expected behavior**
I would expect the two to be equal, with the same input. Also, I'm getting the same result/probability no matter what input I use:

![image](https://github.com/dotnet/machinelearning/assets/105062729/34a77e84-b547-477a-8213-5a472641fb7b)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.