dotnet / dotnet/machinelearning
OneDAL FastForest training has an "Array dimensions exceeded supported range" exception
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**System Information (please complete the following information):**
- OS & Version: Linux Alpine
- ML.NET Version: ML.NET 3.0
- .NET Version: .Net 5.0
**Describe the bug**
```
Array dimensions exceeded supported range. at System.Collections.Generic.List`1.set_Capacity(Int32 value)
at System.Collections.Generic.List`1.AddWithResize(T item)
at Microsoft.ML.OneDal.OneDalUtils.GetTrainData(IChannel channel, Factory cursorFactory, List`1& featuresList, List`1& labelsList, Int32 numberOfFeatures)
at Microsoft.ML.Trainers.FastTree.FastForestBinaryTrainer.TrainCoreOneDal(IChannel ch, Factory cursorFactory, Int32 featureCount)
at Microsoft.ML.Trainers.FastTree.FastForestBinaryTrainer.TrainModelCore(TrainContext context)
at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor)
at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input)
at Microsoft.ML.AutoML.BinaryClassificationRunner.Run(TrialSettings settings)
at Microsoft.ML.AutoML.BinaryClassificationRunner.RunAsync(TrialSettings settings, CancellationToken ct)
at Microsoft.ML.AutoML.AutoMLExperiment.RunAsync(CancellationToken ct)
```
We have found this error in internal ML.net logs a lot of times. Looks liks it's not related to train set size (in the case I have copied this error we have only 20 000 training rows)
Contributor guide
Research direction
Start at Microsoft.ML.OneDal.OneDalUtils.GetTrainData and the FastForestBinaryTrainer.TrainCoreOneDal entry point named in the stack trace. Reproduce with the reported Alpine Linux, ML.NET 3.0, and .NET 5.0 environment, then inspect why the training-data lists reach an unsupported capacity. Done means the reported FastForest training case no longer throws this exception and the regression is covered by an appropriate test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, machine-learning
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100