dotnet / dotnet/machinelearning
AutoML - Concatenated columns should have the same type. Column 'IntColumn2' has type of Single, but expected column type is Byte.
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**System Information (please complete the following information):**
- OS & Version: Windows 11
- ML.NET Version: ML.NET 1.7
- .NET Version: .NET 6 / .NET Standard 2.0
**Describe the bug**
I am trying to use Microsoft.ML.AutoML.MultiClassificationExperiment with preloaded SQLData
```
System.InvalidOperationException: 'Training failed with the exception: System.InvalidOperationException: Concatenated columns should have the same type. Column 'Doc_IntColumn2' has type of Single, but expected column type is Byte.
at Microsoft.ML.Transforms.ColumnConcatenatingEstimator.CheckInputsAndMakeColumn(SchemaShape inputSchema, String name, String[] sources)
at Microsoft.ML.Transforms.ColumnConcatenatingEstimator.GetOutputSchema(SchemaShape inputSchema)
at Microsoft.ML.Data.EstimatorChain`1.GetOutputSchema(SchemaShape inputSchema)
at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input)
at Microsoft.ML.AutoML.SuggestedPipeline.ToEstimator(IDataView trainset, IDataView validationSet)
at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String groupId, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, IChannel logger)'
```
Stacktrace
```
at Microsoft.ML.AutoML.Experiment`2.Execute()
at Microsoft.ML.AutoML.ExperimentBase`2.Execute(ColumnInformation columnInfo, DatasetColumnInfo[] columns, IEstimator`1 preFeaturizer, IProgress`1 progressHandler, IRunner`1 runner)
at Microsoft.ML.AutoML.ExperimentBase`2.ExecuteCrossValSummary(IDataView[] trainDatasets, ColumnInformation columnInfo, IDataView[] validationDatasets, IEstimator`1 preFeaturizer, IProgress`1 progressHandler)
at Microsoft.ML.AutoML.ExperimentBase`2.Execute(IDataView trainData, ColumnInformation columnInformation, IEstimator`1 preFeaturizer, IProgress`1 progressHandler)
at Intense.AI.AutoML.Trainers.MulticlassClassification.RunExperiment(IDataView trainingDataView, RunExperimentDto runExperimentDto) in xxxx.cs:line 63
```
Not working input
```
SELECT
[Name], ISNULL(ShortStringColumn3,'') AS [ShortStringColumn3],
CAST(IntColumn2 as real) as [IntColumn2], CAST(IntColumn3 as real) as [IntColumn3], CAST(ISNULL(IntColumn4,0) as real) as [IntColumn4],
cast(IntColumn5 as real) as [LabelColumn]
FROM Documents
```
```
Name ShortStringColumn3 IntColumn2 IntColumn3 IntColumn4 LabelColumn
--------------- ----------------------- -------------- -------------- -------------- ------------
Administrator 73 0 0 73
User 8 0 1583 8
Administrator 3 0 1583 73
```
But on the other hand I have no problem with AutoML training with this dataset (single + string) as input
Working input
```
SELECT
CAST(Customer as real) as [Customer], cast([Name] as varchar(50)) as Name, [IntColumn21] as [LabelColumn]
FROM Documents
```
```
Customer Name LabelColumn
------------- ---------- -----------
2306 xyz 1
1666 dataaab 1
2323 dataaaa 1
2158 aaaaaac 1
2158 aaaaaab 0
2082 yyyyyy 0
2082 yyyyyy 0
```
**To Reproduce**
Steps to reproduce the behavior:
var loader = MLContext.Data.CreateDatabaseLoader(columns.ToArray());
var dbSource = new DatabaseSource(SqlClientFactory.Instance, connectionString, sqlQuery);
var iDataView = loader.Load(dbSource);
experiment.Execute(trainData: iDataView, labelColumnName: "LabelColumn", progressHandler: progressHandler);

**Expected behavior**
I understand there's problem with connectin string/int columns in input data... (same if int cols are first and strings are later). Why can't allow to auto concatenate all fields?
Contributor guide
Assessment
This issue has not been assessed yet.