dotnet / dotnet/machinelearning
AutoML Binary Classification Experiment run for few second and finished without models
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**System Information (please complete the following information):**
- OS & Version: mcr.microsoft.com/dotnet/sdk:5.0 docker image (Linux Alpine)
- ML.NET Version: 3.0.0-preview.23511.1
- .NET Version: NET 5.0
**Describe the bug**
At this moment we use ML.net 2, but because of the bug fix of https://github.com/dotnet/machinelearning/pull/6571 we have to switch to 3 version of ML.net to train our Binary Classification models (we need Positive Recall optimization metric).
But looks like Binary Classification Experiment is somehow broken in 3 version of ML.net:
var settings = new BinaryExperimentSettings
{
MaxExperimentTimeInSeconds = 30 * 60,
//MaxModels = 10,
OptimizingMetric = BinaryClassificationMetric.PositiveRecall,
MaximumMemoryUsageInMegaByte = 7500,
UseAutoZeroTuner = false
};
ExperimentResult experimentResult = experiment
.Execute(trainDataView, nameof(MlModelRow.Label), nameof(MlModelRow.LearningGroup));
**_We use only FastForest and LightGBM trainers._** On my local PC (Windows 10) it's working great, but in the production docker image (Alpine Linux) the learning is finished after 10-30 seconds with:
> Training time finished without completing a successful trial. Either no trial completed or the metric for all completed trials are NaN or Infinity
I have tried to:
1. Use MaxModels = 10 with MaxExperimentTimeInSeconds
2. Use MaxModels = 10 insted of MaxExperimentTimeInSeconds
3. Turn UseAutoZeroTuner to true
But nothing is working for me. Important point - MLNET_BACKEND is not set so we are not using OneDAL on production or test environment.
Contributor guide
Assessment
This issue has not been assessed yet.