dotnet / dotnet/machinelearning
Saved model returning a different prediction when using AddPredictionEnginePool
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**System Information (please complete the following information):**
- OS & Version: Windows 10
- ML.NET Version: 3.0.0
- .NET Version: 7.0
**Describe the bug**
I'm trying to build something with an API endpoint exposed. While prototyping, I'm essentially following the documentation here: https://learn.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/serve-model-web-api-ml-net
I would expect the following two blocks to give the same result:
(In a console app)
```
var mlContext = new MLContext();
SentimentIssue sampleStatement = new SentimentIssue { Text = "I love this movie!" };
string ModelPath = "path/to/SentimentModel.zip";
ITransformer loadedModel = mlContext.Model.Load(ModelPath, out var modelInputSchema);
var predEngine = mlContext.Model.CreatePredictionEngine(loadedModel);
var resultprediction = predEngine.Predict(sampleStatement);
Console.WriteLine($"=============== Single Prediction ===============");
Console.WriteLine($"Text: {sampleStatement.Text} | Prediction: {(Convert.ToBoolean(resultprediction.Prediction) ? "Positive" : "Negative")} sentiment | Probability of being positive: {resultprediction.Probability} ");
```
Actual output:

In an API, in the Program.cs file:
```
builder.Services.AddPredictionEnginePool().FromFile(modelName: "SentimentAnalysisModel", filePath: "path/to/SentimentModel.zip", watchForChanges: true);
// ...
var predictionHandler =
(PredictionEnginePool predictionEnginePool, SentimentAnalysisModelInput input) =>
{
return predictionEnginePool.Predict(modelName: "SentimentAnalysisModel", input);
};
app.MapPost("/predict", predictionHandler);
app.Run();
```
The output (for example, using Postman with the same input)

**Expected behavior**
I would expect the two to be equal, with the same input. Also, I'm getting the same result/probability no matter what input I use:

Contributor guide
Assessment
This issue has not been assessed yet.