dotnet / dotnet/machinelearning

OneDAL FastForest training has an "Array dimensions exceeded supported range" exception

Open
#6,927 2 comments 0 reactions 0 assignees View on GitHub
area-Native untriaged
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**
- OS & Version: Linux Alpine
- ML.NET Version: ML.NET 3.0
- .NET Version: .Net 5.0

**Describe the bug**

```
Array dimensions exceeded supported range. at System.Collections.Generic.List`1.set_Capacity(Int32 value)
at System.Collections.Generic.List`1.AddWithResize(T item)
at Microsoft.ML.OneDal.OneDalUtils.GetTrainData(IChannel channel, Factory cursorFactory, List`1& featuresList, List`1& labelsList, Int32 numberOfFeatures)
at Microsoft.ML.Trainers.FastTree.FastForestBinaryTrainer.TrainCoreOneDal(IChannel ch, Factory cursorFactory, Int32 featureCount)
at Microsoft.ML.Trainers.FastTree.FastForestBinaryTrainer.TrainModelCore(TrainContext context)
at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor)
at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input)
at Microsoft.ML.AutoML.BinaryClassificationRunner.Run(TrialSettings settings)
at Microsoft.ML.AutoML.BinaryClassificationRunner.RunAsync(TrialSettings settings, CancellationToken ct)
at Microsoft.ML.AutoML.AutoMLExperiment.RunAsync(CancellationToken ct)
```

We have found this error in internal ML.net logs a lot of times. Looks liks it's not related to train set size (in the case I have copied this error we have only 20 000 training rows)

Contributor guide

Open the contributing guide

Research direction

Start at Microsoft.ML.OneDal.OneDalUtils.GetTrainData and the FastForestBinaryTrainer.TrainCoreOneDal entry point named in the stack trace. Reproduce with the reported Alpine Linux, ML.NET 3.0, and .NET 5.0 environment, then inspect why the training-data lists reach an unsupported capacity. Done means the reported FastForest training case no longer throws this exception and the regression is covered by an appropriate test.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, machine-learning
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.