dotnet / dotnet/machinelearning

Improving column purpose detection for sparse datasets

Open
#3,879 0 comments 0 reactions 0 assignees View on GitHub
area-AutoML enhancement Priority:2
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

AutoML does poorly on a few text datasets. For example, a text dataset we benchmark on has an accuracy of 0.60 vs. an expected accuracy of 0.85.

This is caused by us detecting the text columns columns as **categorical** instead of **free text**. For the this dataset, this is due to the text column being 84% blank (a sparsely filled out column).

**To fix:**
We need to detect the column purpose only on the set (non-blank) values.

Recommend subtracting the blank values from `data.Count`:
https://github.com/dotnet/machinelearning/blob/227da9d7db2ce80b073cc64bfd067b04e6189de1/src/Microsoft.ML.AutoML/ColumnInference/PurposeInference.cs#L148-L158

Currently `avgLength`, `cardinalityRatio`, `avgSpaces` are artificially lower due to the missing values.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.