dotnet / dotnet/TorchSharp

Unable to use torch.Generator on CUDA

Open
#1,194 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
C#
Stars
1.9k
Forks
228
PR merge metrics
No merged PRs in 30d

Description

Minimal reproduction:

```c#
var generator = new torch.Generator(42, torch.device("cuda"));
Console.WriteLine(generator.device);
Console.WriteLine(generator.get_state());
var distribution = torch.tensor(new float[] {0.1f, 0.2f, 0.3f, 0.4f}, device: torch.device("cuda"));
var output = torch.multinomial(distribution, num_samples: 1, generator: generator);
Console.WriteLine(output.ToString(true));
```

Output:
```
cuda
[5056], type = Byte, device = cpu
Unhandled exception. System.Runtime.InteropServices.ExternalException (0x80004005): Expected a 'cuda' device type for generator but found 'cpu'
Exception raised from check_generator at /opt/conda/conda-bld/pytorch_1695392067780/work/aten/src/ATen/core/Generator.h:156 (most recent call first):
...(call stack omitted)
```

This also won't work:
```c#
var generator = new torch.Generator(42, torch.device("cuda"));
generator.manual_seed(42);
Console.WriteLine(generator.device);
Console.WriteLine(generator.get_state());
generator.set_state(generator.get_state().cuda());
```

Output:
```
cuda
[5056], type = Byte, device = cpu
terminate called after throwing an instance of 'c10::TypeError'
what(): RNG state must be a torch.ByteTensor
Exception raised from check_rng_state at /opt/conda/conda-bld/pytorch_1695392067780/work/aten/src/ATen/core/Generator.h:181 (most recent call first):
...(call stack omitted)
```

TorchSharp: 0.101.4
libtorch loaded from conda: pytorch 2.1.0 py3.10_cuda12.1_cudnn8.9.2_0

---

Update:

This issue may be more complicated. The equivalent code works in python/pytorch, and the device of state tensor is exactly `cpu` with shape `[16]`. A rolling offset is also used in pytorch.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.