dotnet / dotnet/machinelearning

Cannot retrain the model that it used LbfgsMaximumEntropy algorithms

Open
#5,795 1 comment 1 reaction 0 assignees View on GitHub
area-Trainers Priority:3 question
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

Hi Najeeb
Follow you guide in the last tip, I created the function to retrain the model. I used LbfgsMaximumEntropy algorithms:

// STEP 1: Common data loading configuration
var data = mlContext.Data.LoadFromEnumerable(modelList);

// STEP 2: Common data process configuration with pipeline data transformations
var dataPrepEstimator = mlContext.Transforms.Conversion.MapValueToKey(outputColumnName: "Label", inputColumnName: nameof(PredictModel.EngineerId))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "JobTypeIdFeaturized", inputColumnName: nameof(PredictModel.JobTypeId)))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "CustomerIdFeaturized", inputColumnName: nameof(PredictModel.CustomerId)))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "CustomerTypeIdFeaturized", inputColumnName: nameof(PredictModel.CustomerTypeId)))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "SiteIdFeaturized", inputColumnName: nameof(PredictModel.SiteId)))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "TenantIdFeaturized", inputColumnName: nameof(PredictModel.TenantId)))
.Append(mlContext.Transforms.Concatenate(outputColumnName: "Features", "JobTypeIdFeaturized", "CustomerIdFeaturized",
"CustomerTypeIdFeaturized", "SiteIdFeaturized", "TenantIdFeaturized"));

var lbfgsTrainingPipeLine = dataPrepEstimator.Append(mlContext.MulticlassClassification.Trainers.LbfgsMaximumEntropy("Label", "Features"));
var keyToValuePipeLine = mlContext.Transforms.Conversion.MapValueToKey("PredictedLabel");

ITransformer trainedModel = lbfgsTrainingPipeLine.Fit(data);
ITransformer keyToValueModel = keyToValuePipeLine.Fit(trainedModel.Transform(data));

mlContext.Model.Save(trainedModel, data.Schema, $"{modelPath}\\data_predict.zip");
mlContext.Model.Save(keyToValueModel, trainedModel.Transform(data).Schema, $"{modelPath}\\data_prep_pipeline.zip");

My question is:
1. How can I use the trained model to predict?
var predictionEngine = mlContext.Model.CreatePredictionEngine(trainedModel); => this code got an error.

2. How can I retrain the model with this way? This is my retrain function, and I got an error in step 4.
public static void RetrainModel(MLContext mlContext, IEnumerable modelList, string modelPath)
{
Console.WriteLine("Re-training regression model ...");

// 1. Load data preparation pipeline
ITransformer dataPrepPipeline = mlContext.Model.Load($"{modelPath}\\data_prep_pipeline.zip", out _);

// 2. Load trained model
ITransformer trainedModel = mlContext.Model.Load($"{modelPath}\\data_predict.zip", out _);

var predictor = (trainedModel as TransformerChain).LastTransformer as MulticlassPredictionTransformer;
MaximumEntropyModelParameters originalModelParameters = predictor.Model;

// 3. Load new data
var newData = mlContext.Data.LoadFromEnumerable(modelList);

// 4. Process new data
IDataView transformedNewData = dataPrepPipeline.Transform(newData);

// 5. Retrain model
var retrainedModel = mlContext.MulticlassClassification.Trainers.LbfgsMaximumEntropy("Label", "Features").Fit(transformedNewData, originalModelParameters);

///mlContext.Model.Save(dataPrepPipeline, newData.Schema, $"{modelPath}\\data_prep_pipeline.zip");
mlContext.Model.Save(retrainedModel, transformedNewData.Schema, $"{modelPath}\\data_predict.zip");
}

Please answer when you can.
Thanks and best regards.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.