dotnet / dotnet/machinelearning
GC pause caused by Tuple<int, int> on model load
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**Is your feature request related to a problem? Please describe.**
My app is quite sensitive to GC pauses, so I've been doing allocation profiling. I noticed that a lot of GC pauses are being caused by the usage of `Tuple` within the `Attention` class when the model is loaded.
https://github.com/dotnet/machinelearning/blob/aaf226c7e7c359edf27e663362e928e02c8b9d0f/src/Microsoft.ML.TorchSharp/AutoFormerV2/Attention.cs#L72-L79
**Describe the solution you'd like**
In my own fork, I've switched to using `ValueTuple`, which has removed the allocations and thus GC pauses
**Describe alternatives you've considered**
I don't think there's an easier/cleaner fix for this.
**Additional context**
Using `Microsoft.ML.TorchSharp` version `0.21.0-preview.23266.6`, I observed 240k allocations of `Tuple` during model loading, coming in just after `Tensor`.
After switching to `ValueTuple` on my own fork, these allocations were removed, resulting in a significant reduction in CG pauses during model loading.
Contributor guide
Assessment
This issue has not been assessed yet.