dotnet / dotnet/machinelearning
Suggestion: Make TransformInference public
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
Creating pipelines for big datasets can be complex. Datasets may have hundreds or thousands of columns. Type of columns may vary. Even if the type is text, some may be boolean, some may be categories, short sentences or long paragraphs. Developers would benefit from tools to help build the pipeline based on rules that analyse the contents.
AutoML already has tooling for automating inference. However, TransormInferenceApi is internal to Microsoft.ML. ColumnInference is public, but using it without TransformInference seems difficult.
1) Could this TransformInferenceApi be public? https://github.com/dotnet/machinelearning/blob/04dda55ab0902982b16309c8e151f13a53e9366d/src/Microsoft.ML.AutoML/TransformInference/TransformInferenceApi.cs
2) Or, is there another way to do transform inference from an application referencing the nuget package?
Because the methods are public but the type is internal, I suppose there might have been some discussion and reason for this choice. However, I could not find from Github so maybe it is possible to reconsider. For now, I will make it and related classes public in a private build. I'll update here if any issues.
Related: TransformInference.Experts could benefit from an extension point. For example, a developer could register additional "transform experts" for specific dataset, or to improved general purpose experts. (In general, there could be more extension points so there would be less need for custom builds)
Contributor guide
Assessment
This issue has not been assessed yet.