dotnet / dotnet/machinelearning
AggregateException/InvalidOperationException when training from IEnumerable backed by EF Core Context
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
### System information
- .Net Core 2.2, EF Core 2.2.1, ML.NET 0.9
### Issue
- Read data from SQL DB via EF Core as IEnumerable, pass to pipeline via streaming IDataView
- Get InvalidOperationException from Microsoft.ML.Transforms.RowShufflingTransformer.Cursor.LoopProducerWorker()
- It worked fine in ML.NET 0.8. I can work-around by forcing the trainer to be single-threaded but this makes training in 0.9 much slower than in 0.8 (on my machine).
The EF Core problem could be addressed by allowing multiple dbcontexts to be provided but this is likely to be an issue with any non thread-safe IDataView source...
### Source code / logs
Source at: https://github.com/endintiers/Unearth.Demo.ML.FromDB shows this working in 0.8, failing in 0.9 and a work-around for 0.9 (single-threading the training).
Contributor guide
Assessment
This issue has not been assessed yet.