dotnet / dotnet/machinelearning
Cannot retrain the model that it used LbfgsMaximumEntropy algorithms
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
Hi Najeeb
Follow you guide in the last tip, I created the function to retrain the model. I used LbfgsMaximumEntropy algorithms:
// STEP 1: Common data loading configuration
var data = mlContext.Data.LoadFromEnumerable(modelList);
// STEP 2: Common data process configuration with pipeline data transformations
var dataPrepEstimator = mlContext.Transforms.Conversion.MapValueToKey(outputColumnName: "Label", inputColumnName: nameof(PredictModel.EngineerId))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "JobTypeIdFeaturized", inputColumnName: nameof(PredictModel.JobTypeId)))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "CustomerIdFeaturized", inputColumnName: nameof(PredictModel.CustomerId)))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "CustomerTypeIdFeaturized", inputColumnName: nameof(PredictModel.CustomerTypeId)))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "SiteIdFeaturized", inputColumnName: nameof(PredictModel.SiteId)))
.Append(mlContext.Transforms.Text.FeaturizeText(outputColumnName: "TenantIdFeaturized", inputColumnName: nameof(PredictModel.TenantId)))
.Append(mlContext.Transforms.Concatenate(outputColumnName: "Features", "JobTypeIdFeaturized", "CustomerIdFeaturized",
"CustomerTypeIdFeaturized", "SiteIdFeaturized", "TenantIdFeaturized"));
var lbfgsTrainingPipeLine = dataPrepEstimator.Append(mlContext.MulticlassClassification.Trainers.LbfgsMaximumEntropy("Label", "Features"));
var keyToValuePipeLine = mlContext.Transforms.Conversion.MapValueToKey("PredictedLabel");
ITransformer trainedModel = lbfgsTrainingPipeLine.Fit(data);
ITransformer keyToValueModel = keyToValuePipeLine.Fit(trainedModel.Transform(data));
mlContext.Model.Save(trainedModel, data.Schema, $"{modelPath}\\data_predict.zip");
mlContext.Model.Save(keyToValueModel, trainedModel.Transform(data).Schema, $"{modelPath}\\data_prep_pipeline.zip");
My question is:
1. How can I use the trained model to predict?
var predictionEngine = mlContext.Model.CreatePredictionEngine(trainedModel); => this code got an error.
2. How can I retrain the model with this way? This is my retrain function, and I got an error in step 4.
public static void RetrainModel(MLContext mlContext, IEnumerable modelList, string modelPath)
{
Console.WriteLine("Re-training regression model ...");
// 1. Load data preparation pipeline
ITransformer dataPrepPipeline = mlContext.Model.Load($"{modelPath}\\data_prep_pipeline.zip", out _);
// 2. Load trained model
ITransformer trainedModel = mlContext.Model.Load($"{modelPath}\\data_predict.zip", out _);
var predictor = (trainedModel as TransformerChain).LastTransformer as MulticlassPredictionTransformer;
MaximumEntropyModelParameters originalModelParameters = predictor.Model;
// 3. Load new data
var newData = mlContext.Data.LoadFromEnumerable(modelList);
// 4. Process new data
IDataView transformedNewData = dataPrepPipeline.Transform(newData);
// 5. Retrain model
var retrainedModel = mlContext.MulticlassClassification.Trainers.LbfgsMaximumEntropy("Label", "Features").Fit(transformedNewData, originalModelParameters);
///mlContext.Model.Save(dataPrepPipeline, newData.Schema, $"{modelPath}\\data_prep_pipeline.zip");
mlContext.Model.Save(retrainedModel, transformedNewData.Schema, $"{modelPath}\\data_predict.zip");
}
Please answer when you can.
Thanks and best regards.
Contributor guide
Assessment
This issue has not been assessed yet.