dotnet / dotnet/machinelearning

Parallelism: trouble saturating CPU when making predictions

Open
#6,702 2 comments 0 reactions 0 assignees View on GitHub
area-Core needs-author-action untriaged
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**
- OS & Version: Windows Server 2019
- ML.NET Version: ML.NET 1.6.
- .NET Version: .NET 7.0

**Describe the bug**
I run prediction in Parallel.ForEach loop on 16 core machine. The CPU consumption usually stabilizes near 10-12% even is max parallelism would be 16.

I would like to understand what is might be the bottleneck. Are there some parts of Microsoft.ML that may be limiting, such as locks(?) or parallel data structures that do it internally?

Memory hard Faults usually <10, max < 100. I am caching models and dataset, about 90% RAM used. Disk 0-20% Highest active time.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.