dotnet / dotnet/machinelearning

Unable to load KMeans model as original data type

Open
#6,079 2 comments 0 reactions 0 assignees View on GitHub
area-Core
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**
- OS & Version: Windows 10
- ML.NET Version: ML.Net 1.7.0
- .NET Version: .Net 6.0

**Describe the bug**
After fitting a KMeans model, the trained model is of type TransformerChain>. After saving the model to a file using mlContext.Model.Save and reloading it using mlContext.Model.Load, there is no way to get the model back to TransformerChain>. The initial object loaded is of type TransformerChain. If you look at the object.LastTransformer, it is of type ClusteringPredictionTransformer>>. Casting either the loaded object or the LastTransformer gives a System.InvalidCastException. Using mlContext.Model.Load(...) as ... gives you null. Leaving LastTransformer as ITransformer does not provide you with the "Model" property.

**To Reproduce**

Code below for simple Console app, based on MS KMeans example.

```
using System;
using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Trainers;
using System.Collections.Generic;
using System.IO;

namespace KMeansSample
{
internal class Program
{
static void Main(string[] args)
{
// Create a new context for ML.NET operations. It can be used for
// exception tracking and logging, as a catalog of available operations
// and as the source of randomness. Setting the seed to a fixed number
// in this example to make outputs deterministic.
var mlContext = new MLContext(seed: 0);

// Create a list of training data points.
var dataPoints = GenerateRandomDataPoints(1000, 0);

var _model = mlContext.Clustering.Trainers.KMeans(numberOfClusters: 3, featureColumnName: "Features");

// Convert the list of data points to an IDataView object, which is
// consumable by ML.NET API.
IDataView trainingData = mlContext.Data.LoadFromEnumerable(dataPoints);
var dataProcessPipeline = mlContext.Transforms.Concatenate("Features", nameof(DataPoint.val1),
nameof(DataPoint.val2),
nameof(DataPoint.val3),
nameof(DataPoint.val4)).AppendCacheCheckpoint(mlContext);
var trainingPipeline = dataProcessPipeline.Append(_model);

var trainedModel = trainingPipeline.Fit(trainingData);

mlContext.Model.Save(trainedModel, trainingData.Schema, "C:\\temp\\trainedModel.ms");

//This version will produce a System.InvalidCastException
using (var stream = new FileStream("C:\\temp\\trainedModel.ms", FileMode.Open, FileAccess.Read, FileShare.Read))
{
var loadedModel = (TransformerChain>)(mlContext.Model.Load(stream, out var modelInputSchema1));
var lastTransformer = loadedModel.LastTransformer;
var modelParameters = lastTransformer.Model;
}

//This version will produce a null reference exception
using (var stream = new FileStream("C:\\temp\\trainedModel.ms", FileMode.Open, FileAccess.Read, FileShare.Read))
{
var loadedModel = mlContext.Model.Load(stream, out var modelInputSchema1) as TransformerChain>;
var lastTransformer = loadedModel.LastTransformer;
var modelParameters = lastTransformer.Model;
}

//This version gives a compiler error

using (var stream = new FileStream("C:\\temp\\trainedModel.ms", FileMode.Open, FileAccess.Read, FileShare.Read))
{
var loadedModel = mlContext.Model.Load(stream, out var modelInputSchema1) as TransformerChain ;
var lastTransformer = loadedModel.LastTransformer;
var modelParameters = lastTransformer.Model;
}

//This version also gives a compiler error
using (var stream = new FileStream("C:\\temp\\trainedModel.ms", FileMode.Open, FileAccess.Read, FileShare.Read))
{
var loadedModel = mlContext.Model.Load(stream, out var modelInputSchema1);
var lastTransformer = loadedModel.LastTransformer;
var modelParameters = lastTransformer.Model;
}

}

private static IEnumerable GenerateRandomDataPoints(int count,
int seed = 0)
{
var random = new Random(seed);
float randomFloat() => (float)random.NextDouble();
for (int i = 0; i < count; i++)
{

yield return new DataPoint
{
val1 = randomFloat(),
val2 = randomFloat(),
val3 = randomFloat(),
val4 = randomFloat()

};
}
}

// Example with label and 50 feature values. A data set is a collection of
// such examples.
private class DataPoint
{

public float val1 { get; set; }
public float val2 { get; set; }
public float val3 { get; set; }
public float val4 { get; set; }


}

// Class used to capture predictions.
private class Prediction
{
// Original label (not used during training, just for comparison).
public uint Label { get; set; }
// Predicted label from the trainer.
public uint PredictedLabel { get; set; }
}

// Pretty-print of ClusteringMetrics object.
private static void PrintMetrics(ClusteringMetrics metrics)
{
Console.WriteLine($"Normalized Mutual Information: " +
$"{metrics.NormalizedMutualInformation:F2}");

Console.WriteLine($"Average Distance: " +
$"{metrics.AverageDistance:F2}");

Console.WriteLine($"Davies Bouldin Index: " +
$"{metrics.DaviesBouldinIndex:F2}");
}
}
}

```

**Expected behavior**
Able to load object back into the same type as the object that was saved.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.