dotnet / dotnet/TorchSharp

torch.deleters performance issue on .NET framework cpu target

Open
#1,435 6 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
C#
Stars
1.9k
Forks
228
PR merge metrics
No merged PRs in 30d

Description

I discovered this performance issue while using the MNIST dataset.

This only happens with the combination of .NET framework and cpu target. It does not happen with .NET target or cuda.

This is a minimal reproducible source.

```
// match to MNIST dataset size
var size = 70000;

var tensors = new List(size);

var dev = new torch.Device("cpu");

for (int i = 0; i < size; ++i)
{
tensors.Add(torch.tensor(new[] { 1.0f }, device: dev));
}

Console.WriteLine(tensors.Count);

foreach (var tensor in tensors)
{
tensor.Dispose();
}

tensors.Clear();

Console.WriteLine(tensors.Count);
```

The profiler indicates ConcurrentDictionary.TryAdd() and ConcurrentDictionary.TryRemove(), but it looks like MulticastDelegate.Equals() is the problem.

![Image](https://github.com/user-attachments/assets/5623e6a6-d3be-4f9d-8ee6-f803a3ecd752)

![Image](https://github.com/user-attachments/assets/04403e20-7ad6-4e41-9a41-67210016ee66)

For .NET framework and cuda combination, it seems like removed directly on _tensor_generic.

![Image](https://github.com/user-attachments/assets/4901c667-85e8-4ffd-a42e-fe290edc2143)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.