dotnet / dotnet/machinelearning

AutoML 2 is way worse than 1.7.1 (for me)

Open
#6,552 13 comments 0 reactions 1 assignee Claimed by @LittleLittleCloud View on GitHub
area-AutoML
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

Win10 / **ML.NET 1.7.1 vs. 2.0.0** / .NET Framework 4.8

AutoML 2.0 is way worse for me than the previous 1.7.1 release. I tried using the Featurizer or even removing completely and doing it all by hand -- in 2 days of fiddling I can not create a model that is anywhere close to that created with the old CreateRegressionExperiment() version of the previous release.
![image](https://user-images.githubusercontent.com/10066158/215285187-78616615-cc02-4aac-9f33-8c5162b269e0.png)

**To Reproduce**
Steps to reproduce the behavior:
For 2.0 (where the problem is) I used the same code as this sample (but with my objects): https://github.com/dotnet/machinelearning-samples/tree/main/samples/csharp/getting-started/MLNET2/AutoMLAdvanced

```
//Define pipeline
SweepablePipeline pipeline =
ctx.Auto().Featurizer(data, columnInformation: columnInference.ColumnInformation)
.Append(ctx.Auto().Regression(labelColumnName: columnInference.ColumnInformation.LabelColumnName, useLgbm: false));

// Create AutoML experiment
AutoMLExperiment experiment = ctx.Auto().CreateExperiment();

// Configure experiment
experiment
.SetPipeline(pipeline)
.SetRegressionMetric(RegressionMetric.RSquared, labelColumn: columnInference.ColumnInformation.LabelColumnName)
.SetTrainingTimeInSeconds(60)
.SetGridSearchTuner()
.SetDataset(trainValidationData);

// Run experiment
var cts = new CancellationTokenSource();
TrialResult experimentResults = await experiment.RunAsync(cts.Token);
```

I also unwound the featurizer and did all the same steps by hand and they worked with 1.7.1.

**Expected behavior**
To be able to train a model that works as well as the last version.

**Additional context**
NOTE: I had all kinds of different versions on my machine and completely uninstalled Visual Studio, deleted the directory, etc.

_Maybe relevant?_

* Now, after re-installing VS and adding ML.net, I no longer have the ability to edit notebooks (.ipynb).

* Sometimes, when playing with the ML.NET Model Builder 2022 (16.13.9.2235601) and the same data, I don't get a Next button with my data. [maybe there's something with my data that causes a problem with the 2.0 code?]
![image](https://user-images.githubusercontent.com/10066158/215289551-df437b02-26c8-48d4-a5f0-243a5f59de2f.png)

**ANY IDEAS WHERE I CAN DEBUG MORE? OR TELL ME WHAT YOU WOULD LIKE TO HAVE ME CAN SHARE SO THAT I CAN BE MORE HELPFUL.**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.