Unable to use torch.Generator on CUDA
- Dominant language
- C#
- Stars
- 1.9k
- Forks
- 228
- PR merge metrics
- No merged PRs in 30d
Description
Minimal reproduction:
```c#
var generator = new torch.Generator(42, torch.device("cuda"));
Console.WriteLine(generator.device);
Console.WriteLine(generator.get_state());
var distribution = torch.tensor(new float[] {0.1f, 0.2f, 0.3f, 0.4f}, device: torch.device("cuda"));
var output = torch.multinomial(distribution, num_samples: 1, generator: generator);
Console.WriteLine(output.ToString(true));
```
Output:
```
cuda
[5056], type = Byte, device = cpu
Unhandled exception. System.Runtime.InteropServices.ExternalException (0x80004005): Expected a 'cuda' device type for generator but found 'cpu'
Exception raised from check_generator at /opt/conda/conda-bld/pytorch_1695392067780/work/aten/src/ATen/core/Generator.h:156 (most recent call first):
...(call stack omitted)
```
This also won't work:
```c#
var generator = new torch.Generator(42, torch.device("cuda"));
generator.manual_seed(42);
Console.WriteLine(generator.device);
Console.WriteLine(generator.get_state());
generator.set_state(generator.get_state().cuda());
```
Output:
```
cuda
[5056], type = Byte, device = cpu
terminate called after throwing an instance of 'c10::TypeError'
what(): RNG state must be a torch.ByteTensor
Exception raised from check_rng_state at /opt/conda/conda-bld/pytorch_1695392067780/work/aten/src/ATen/core/Generator.h:181 (most recent call first):
...(call stack omitted)
```
TorchSharp: 0.101.4
libtorch loaded from conda: pytorch 2.1.0 py3.10_cuda12.1_cudnn8.9.2_0
---
Update:
This issue may be more complicated. The equivalent code works in python/pytorch, and the device of state tensor is exactly `cpu` with shape `[16]`. A rolling offset is also used in pytorch.
Contributor guide
Assessment
This issue has not been assessed yet.