dotnet / dotnet/machinelearning

Suggestion: Make TransformInference public

Open
#6,058 2 comments 0 reactions 0 assignees View on GitHub
area-AutoML enhancement
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

Creating pipelines for big datasets can be complex. Datasets may have hundreds or thousands of columns. Type of columns may vary. Even if the type is text, some may be boolean, some may be categories, short sentences or long paragraphs. Developers would benefit from tools to help build the pipeline based on rules that analyse the contents.

AutoML already has tooling for automating inference. However, TransormInferenceApi is internal to Microsoft.ML. ColumnInference is public, but using it without TransformInference seems difficult.

1) Could this TransformInferenceApi be public? https://github.com/dotnet/machinelearning/blob/04dda55ab0902982b16309c8e151f13a53e9366d/src/Microsoft.ML.AutoML/TransformInference/TransformInferenceApi.cs
2) Or, is there another way to do transform inference from an application referencing the nuget package?

Because the methods are public but the type is internal, I suppose there might have been some discussion and reason for this choice. However, I could not find from Github so maybe it is possible to reconsider. For now, I will make it and related classes public in a private build. I'll update here if any issues.

Related: TransformInference.Experts could benefit from an extension point. For example, a developer could register additional "transform experts" for specific dataset, or to improved general purpose experts. (In general, there could be more extension points so there would be less need for custom builds)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.