dotnet / dotnet/machinelearning

Ignore Columns ColumnInference

Open
#6,777 0 comments 0 reactions 0 assignees View on GitHub
area-AutoML enhancement
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

When using `InferColumns`, there is now way to specify which columns to exclude.

```csharp
// Define data path
var dataPath = Path.GetFullPath(@"../Data/issues_train.tsv");

// Infer column information
ColumnInferenceResults columnInference =
ctx.Auto().InferColumns(dataPath, separatorChar: '\t', labelColumnName: "Area", groupColumns: false);
```

That means that you have to do it manually.

```csharp
var columnsToExclude = new[]{"ID","Title"};

columnInference.TextLoaderOptions.Columns =
columnInference.TextLoaderOptions.Columns
.Where(col => !columnsToExclude.Contains(col.Name)).ToArray();

columnInference.ColumnInformation.NumericColumnNames.Remove("ID");
columnInference.ColumnInformation.TextColumnNames.Remove("Title");
```

Since the TextLoaderOptions and ColumnInformation are used downstream

```csharp
// Create text loader
TextLoader loader = ctx.Data.CreateTextLoader(columnInference.TextLoaderOptions);

// Load data into IDataView
IDataView data = loader.Load(dataPath);

SweepablePipeline pipeline =
ctx.Auto().Featurizer(data, columnInformation: columnInference.ColumnInformation)
.Append(ctx.Transforms.Conversion.MapValueToKey(columnInference.ColumnInformation.LabelColumnName))
.Append(ctx.Auto().MultiClassification(labelColumnName: columnInference.ColumnInformation.LabelColumnName))
.Append(ctx.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
```

In the InferColumns API, it might be the best place to set this value

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.