dotnet / dotnet/machinelearning-samples
I sometimes get this with the regression example: System.InvalidOperationException: Cannot hold covariance matrix in memory
- Dominant language
- PowerShell
- Stars
- 4.7k
- Forks
- 2.7k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 1
Description
Any idea what could be causing this. My training data is not very big. 40K rows.
Running AutoML regression experiment for 60 seconds...
| Trainer RSquared Absolute-loss Squared-loss RMS-loss Duration |
|1 LightGbmRegression 0.9930 140404.81 42178224023.09 205373.38 9.2 |
|2 FastTreeRegression 0.9934 136725.85 39940180886.08 199850.40 8.8 |
|3 FastTreeTweedieRegression 0.9913 142834.10 52816241323.19 229817.84 9.2 |
|4 FastForestRegression 0.9091 538589.21 551116053939.22 742371.91 9.4 |
Exception during AutoML iteration: System.InvalidOperationException: Cannot hold covariance matrix in memory with 94459 features
at Microsoft.ML.Trainers.OlsTrainer.TrainCore(IChannel ch, Factory cursorFactory, Int32 featureCount)
at Microsoft.ML.Trainers.OlsTrainer.TrainModelCore(TrainContext context)
at Microsoft.ML.Trainers.TrainerEstimatorBase`2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor)
at Microsoft.ML.Trainers.TrainerEstimatorBase`2.Fit(IDataView input)
at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input)
at Microsoft.ML.AutoML.RunnerUtil.TrainAndScorePipeline[TMetrics](MLContext context, SuggestedPipeline pipeline, IDataView trainData, IDataView validData, String groupId, String labelColumn, IMetricsAgent`1 metricsAgent, ITransformer preprocessorTransform, FileInfo modelFileInfo, DataViewSchema modelInputSchema, IChannel logger)
|6 LightGbmRegression 0.9295 447071.45 427662770504.29 653959.30 9.4 |
|7 FastTreeRegression 0.2663 1336968.13 4450480773603.34 2109616.26 7.0 |
|8 FastTreeTweedieRegression 0.9923 133275.19 46493327388.81 215623.11 12.2 |
Top models ranked by R-Squared --
| Trainer RSquared Absolute-loss Squared-loss RMS-loss Duration |
|1 FastTreeRegression 0.9934 136725.85 39940180886.08 199850.40 8.8 |
|2 LightGbmRegression 0.9930 140404.81 42178224023.09 205373.38 9.2 |
|3 FastTreeTweedieRegression 0.9923 133275.19 46493327388.81 215623.11 12.2 |
Contributor guide
Assessment
This issue has not been assessed yet.