SdcaMaximumEntropy and SdcaNonCalibrated take too long to train

### Type of issue

Other (describe below)

### Description

Previously, I created a dynamic code to preprocess the features columns as follows:

int i = 0;

/* Create the pipeline. */

/* Preprocess the numeric column. */

/* Replace missing data in the numerical column, and put that column in a new column ("FeaturesComplete"). */

var Pipeline = MLContext.Transforms.ReplaceMissingValues(outputColumnName: "FeaturesComplete", inputColumnName: NumColName)
/* Normalize data (between 0 and 1) in the numerical column ("FeaturesComplete"), and put it in a new concatenated column ("FeaturesComplete&Normalized"). */
.Append(MLContext.Transforms.NormalizeMeanVariance(outputColumnName: "FeaturesComplete&Normalized", inputColumnName: "FeaturesComplete"))
/* Concatenate the preprocessed numerical column into 1 final features column ("FeaturesFinal"). */
.Append(MLContext.Transforms.Concatenate("FeaturesFinal", "FeaturesComplete&Normalized"));

/* Preprocess the categorical columns. */

/* The presence of 1 or more categorical columns. */
if (CatCols.Count >= 1)
{
/* This loop is used to transform all the categorical columns in the training dataset. */
for (i = 0; i < CatColsArray.Length; i++)
{
/* Encode the categorical columns in CatColsArray, and put them in a new column with the same name. */
Pipeline = Pipeline.Append(MLContext.Transforms.Categorical.OneHotEncoding(outputColumnName: CatColsArray[i].ToString(), inputColumnName: CatColsArray[i].ToString()))
/* Concatenate the preprocessed categorical column i into the final features column ("FeaturesFinal"). */
.Append(MLContext.Transforms.Concatenate("FeaturesFinal", CatColsArray[i].ToString()));
}
}

/* Preprocess the text columns. */

/* The presence of 1 or more text columns. */
if (TextCols.Count >= 1)
{
/* This loop is used to transform all the text columns in the training dataset. */
for (i = 0; i < TextColsArray.Length; i++)
{
/* Featurize the text columns in TextColsArray, and put them in a new column with the same name. */
Pipeline = Pipeline.Append(MLContext.Transforms.Text.FeaturizeText(outputColumnName: TextColsArray[i].ToString(), inputColumnName: TextColsArray[i].ToString()))
/* Concatenate the preprocessed text column i into the final features column ("FeaturesFinal"). */
.Append(MLContext.Transforms.Concatenate("FeaturesFinal", TextColsArray[i].ToString()));
}
}

var PipelineFinal = Pipeline.Append(MLContext.Transforms.Conversion.MapValueToKey(outputColumnName: "Label", inputColumnName: LabelColName));
return PipelineFinal;

The above code works fine, however, I tried to simplify it as follows:

var pipeline = MLContext.Transforms.Concatenate("FeaturesFinal", NumColName)
.Append(MLContext.Transforms.Conversion.MapValueToKey(outputColumnName: "Label", inputColumnName: LabelColName));
return pipeline;

But the trainers in this case (SdcaMaximumEntropy and SdcaNonCalibrated) take too long to finish training (about 30 minutes compared to 1 minute using the dynamic preprocessing code above!!! I do not understand why!

The training code (for your reference):

ITransformer[] Models = new ITransformer[3];

/* Build and train three models using the Sdca Maximum Entropy, Sdca Non-calibrated, and the Naive Bayes Multiclass classification trainers to choose the best performing model. */
var TrainingPipeline1 = Pipeline.Append(MLContext.MulticlassClassification.Trainers.SdcaMaximumEntropy(labelColumnName: "Label", featureColumnName: "FeaturesFinal"))
.Append(MLContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
Models[0] = TrainingPipeline1.Fit(TrainingDataView);

var TrainingPipeline2 = Pipeline.Append(MLContext.MulticlassClassification.Trainers.SdcaNonCalibrated(labelColumnName: "Label", featureColumnName: "FeaturesFinal"))
.Append(MLContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
Models[1] = TrainingPipeline2.Fit(TrainingDataView);

var TrainingPipeline3 = Pipeline.Append(MLContext.MulticlassClassification.Trainers.NaiveBayes(labelColumnName: "Label", featureColumnName: "FeaturesFinal"))
.Append(MLContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
Models[2] = TrainingPipeline3.Fit(TrainingDataView);

return Models;

Please note that adding .AppendCacheCheckpoint(MLContext) does not make any difference. Please advise. Thank you.

### Page URL

https://learn.microsoft.com/en-us/dotnet/machine-learning/tutorials/github-issue-classification

### Content source URL

https://github.com/dotnet/docs/blob/main/docs/machine-learning/tutorials/github-issue-classification.md

### Document Version Independent Id

8a16b9e9-b5cb-a55e-2135-191bc62c1656

### Article author

@luisquintanilla

### Metadata

* ID: 77fc5686-2288-d607-8136-3146ed02dbf6
* Service: **dotnet-ml**