dotnet / dotnet/machinelearning

AutoML Binary Classification Experiment run for few second and finished without models

Open
#6,882 5 comments 0 reactions 0 assignees View on GitHub
area-AutoML untriaged
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**
- OS & Version: mcr.microsoft.com/dotnet/sdk:5.0 docker image (Linux Alpine)
- ML.NET Version: 3.0.0-preview.23511.1
- .NET Version: NET 5.0

**Describe the bug**
At this moment we use ML.net 2, but because of the bug fix of https://github.com/dotnet/machinelearning/pull/6571 we have to switch to 3 version of ML.net to train our Binary Classification models (we need Positive Recall optimization metric).

But looks like Binary Classification Experiment is somehow broken in 3 version of ML.net:

var settings = new BinaryExperimentSettings
{
MaxExperimentTimeInSeconds = 30 * 60,
//MaxModels = 10,
OptimizingMetric = BinaryClassificationMetric.PositiveRecall,
MaximumMemoryUsageInMegaByte = 7500,
UseAutoZeroTuner = false
};

ExperimentResult experimentResult = experiment
.Execute(trainDataView, nameof(MlModelRow.Label), nameof(MlModelRow.LearningGroup));

**_We use only FastForest and LightGBM trainers._** On my local PC (Windows 10) it's working great, but in the production docker image (Alpine Linux) the learning is finished after 10-30 seconds with:

> Training time finished without completing a successful trial. Either no trial completed or the metric for all completed trials are NaN or Infinity

I have tried to:

1. Use MaxModels = 10 with MaxExperimentTimeInSeconds
2. Use MaxModels = 10 insted of MaxExperimentTimeInSeconds
3. Turn UseAutoZeroTuner to true

But nothing is working for me. Important point - MLNET_BACKEND is not set so we are not using OneDAL on production or test environment.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.