pytorch / pytorch/pytorch

[dynamo] dynamic=True reuses a stale random.Random value after shape specialization

Open
#196,437 0 comments 0 reactions 0 assignees View on GitHub
bot-triaged high priority module: correctness (silent) module: dynamic shapes module: dynamo oncall: pt2 release triage triaged
Dominant language
Python
Stars
103k
Forks
29.5k
PR merge metrics
PR metrics pending

Description

### 🐛 Describe the bug

I found a `torch.compile` wrong-code case involving a persistent
`random.Random` object and dynamic shapes. Given the input-shape history
`[1] -> [1, 5] -> [2, 2] -> [2, 3]`, eager execution consumes the first four
`randint(1, 100)` results (`38, 4, 23, 1`). With `dynamic=True`, the compiled
callable instead uses `38, 4, 23, 23`: the fourth call repeats the value from
the third call and does not advance the Python RNG state.

This reproduces with both `backend="eager"` and `backend="inductor"`, on CPU
and CUDA, and with `fullgraph=False` or `fullgraph=True`. The corresponding
`dynamic=False, fullgraph=False` controls pass. This localizes the behavior to
the Dynamo dynamic-shape/cache path rather than Inductor code generation or
CUDA arithmetic.

The reproducer controls numerical precision and RNG equivalence as follows:

- it does not override tolerances;
- it does not compare max/min indices;
- it calls `torch._dynamo.utils.same` with an eager fp64 reference;
- it sets `torch._inductor.config.fallback_random = True`;
- it resets the PyTorch RNG and creates an equivalently seeded Python RNG
before eager, fp64-reference, and compiled runs.

## Minimal reproducer

```python
import random

import torch
from torch._dynamo.utils import same

DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
SEED = 123456
EVENTS = (
([1], [0.8404]),
([1, 5], [0.4816, -0.5391, -0.8346, 0.8489, 0.7063]),
([2, 2], [-0.8071, 0.4757, -0.7567, -0.4887]),
([2, 3], [0.2015, -0.3666, -0.5008, -0.7583, 0.267, 0.5472]),
)

def fn(value, rng):
return value + rng.randint(1, 100)

def run(callable_fn, dtype):
torch.manual_seed(SEED)
if torch.cuda.is_available():
torch.cuda.manual_seed_all(SEED)
rng = random.Random(SEED)
outputs = []
for shape, values in EVENTS:
value = torch.tensor(values, dtype=dtype, device=DEVICE).reshape(shape)
outputs.append(callable_fn(value, rng))
return outputs, rng.getstate()

def offsets(outputs):
return [
round(output.flatten()[0].item() - values[0], 6)
for output, (_, values) in zip(outputs, EVENTS)
]

torch._inductor.config.fallback_random = True

eager, eager_rng_state = run(fn, torch.float32)
fp64_ref, _ = run(fn, torch.float64)

torch.compiler.reset()
compiled_fn = torch.compile(
fn,
backend="inductor", # "eager" fails in the same way
dynamic=True,
fullgraph=False,
)
compiled, compiled_rng_state = run(compiled_fn, torch.float32)

outputs_match = same(eager, compiled, fp64_ref=fp64_ref)
rng_state_matches = eager_rng_state == compiled_rng_state

print(f"eager offsets: {offsets(eager)}")
print(f"compiled offsets: {offsets(compiled)}")
print(f"outputs_match={outputs_match}")
print(f"rng_state_matches={rng_state_matches}")

assert outputs_match and rng_state_matches
```

## Ablation

No compile `mode` argument was supplied (the default mode was used).

| Device | Backend | `dynamic` | `fullgraph` | Result |
|---|---|---:|---:|---|
| CPU | eager | false | false | pass |
| CPU | eager | true | false | **wrong result + stale RNG state** |
| CPU | eager | true | true | **wrong result + stale RNG state** |
| CUDA | eager | false | false | pass |
| CUDA | eager | true | false | **wrong result + stale RNG state** |
| CUDA | eager | true | true | **wrong result + stale RNG state** |
| CPU | inductor | false | false | pass |
| CPU | inductor | true | false | **wrong result + stale RNG state** |
| CPU | inductor | true | true | **wrong result + stale RNG state** |
| CUDA | inductor | false | false | pass |
| CUDA | inductor | true | false | **wrong result + stale RNG state** |
| CUDA | inductor | true | true | **wrong result + stale RNG state** |

## Trace

I will attach the complete `TORCH_TRACE` log (not only a generated HTML
index). The trace contains three Dynamo compilations for the four calls: the
first rank-1 input, the first rank-2 input, and the `[2, 2]` specialization.
There is no fourth compilation for `[2, 3]`, which is consistent with the
dynamic artifact being reused when the stale `23` is observed.

### Error logs

```text
E0909 19:10:05.444000 2605672 torch/_dynamo/utils.py:3519]
RMSE (res-fp64): 22.00000, (ref-fp64): 0.00000 and shape=torch.Size([2, 3]).
res.dtype: torch.float32, multiplier: 3.000000, tol: 0.000100,
use_larger_multiplier_for_smaller_tensor: 0
torch=2.12.1+cu130, device=cuda, backend=inductor
dynamic=True, fullgraph=False
eager offsets: [38.000001, 4.0, 23.0, 1.0]
compiled offsets: [38.000001, 4.0, 23.0, 23.0]
outputs_match=False
rng_state_matches=False
Traceback (most recent call last):
File "issue_reproducer.py", line 74, in
assert outputs_match and rng_state_matches
AssertionError
```

The incorrect fourth tensor differs from eager by exactly `22` in every
element because the compiled path adds `23` instead of the next Python RNG
draw, `1`.

### Versions

```text
Collecting environment information...
PyTorch version: 2.12.1+cu130
Is debug build: False
CUDA used to build PyTorch: 13.0
ROCM used to build PyTorch: N/A

OS: Ubuntu 24.04.4 LTS (x86_64)
GCC version: (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.39

Python version: 3.12.13 | packaged by Anaconda, Inc. | (main, Jul 9 2026, 14:38:16) [GCC 14.3.0] (64-bit runtime)
Python platform: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.39
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to:
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 591.86
cuDNN version: Could not collect
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Caching allocator config: N/A

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 28
On-line CPU(s) list: 0-27
Vendor ID: GenuineIntel
Model name: Intel(R) Core(TM) i7-14700K
CPU family: 6
Model: 183
Thread(s) per core: 2
Core(s) per socket: 14
Socket(s): 1
Stepping: 1
BogoMIPS: 6835.19
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni vnmi umip waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize flush_l1d arch_capabilities
Virtualization: VT-x
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 672 KiB (14 instances)
L1i cache: 448 KiB (14 instances)
L2 cache: 28 MiB (14 instances)
L3 cache: 33 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-27
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Vulnerable: No microcode
Vulnerability Retbleed: Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI BHI_DIS_S
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] nvidia-cublas==13.1.1.3
[pip3] nvidia-cuda-cupti==13.0.85
[pip3] nvidia-cuda-nvrtc==13.0.88
[pip3] nvidia-cuda-runtime==13.0.96
[pip3] nvidia-cudnn-cu13==9.20.0.48
[pip3] nvidia-cufft==12.0.0.61
[pip3] nvidia-curand==10.4.0.35
[pip3] nvidia-cusolver==12.0.4.66
[pip3] nvidia-cusparse==12.6.3.3
[pip3] nvidia-cusparselt-cu13==0.8.1
[pip3] nvidia-nccl-cu13==2.29.7
[pip3] nvidia-nvjitlink==13.0.88
[pip3] nvidia-nvtx==13.0.85
[pip3] torch==2.12.1+cu130
[pip3] triton==3.7.1
[conda] nvidia-cublas 13.1.1.3 pypi_0 pypi
[conda] nvidia-cuda-cupti 13.0.85 pypi_0 pypi
[conda] nvidia-cuda-nvrtc 13.0.88 pypi_0 pypi
[conda] nvidia-cuda-runtime 13.0.88 pypi_0 pypi
[conda] nvidia-cudnn-cu13 9.20.0.48 pypi_0 pypi
[conda] nvidia-cufft 12.0.0.61 pypi_0 pypi
[conda] nvidia-curand 10.4.0.35 pypi_0 pypi
[conda] nvidia-cusolver 12.0.4.66 pypi_0 pypi
[conda] nvidia-cusparse 12.6.3.3 pypi_0 pypi
[conda] nvidia-cusparselt-cu13 0.8.1 pypi_0 pypi
[conda] nvidia-nccl-cu13 2.29.7 pypi_0 pypi
[conda] nvidia-nvjitlink 13.0.88 pypi_0 pypi
[conda] nvidia-nvtx 13.0.85 pypi_0 pypi
[conda] torch 2.12.1 pypi_0 pypi
[conda] torchvision 0.27.1 pypi_0 pypi
[conda] triton 3.7.1 pypi_0 pypi
```

cc @ezyang @gchanan @kadeng @msaroufim @chauhang @penguinwu @bobrenjc93 @aditvenk @laithsakka @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @amjames @jataylo @azahed98

Contributor guide

Open the contributing guide

Research direction

Start by running the provided torch.compile reproducer with torch._dynamo and the listed dynamic-shape and backend combinations. Inspect the Dynamo dynamic-shape/cache path and the three compilations described in the TORCH_TRACE log. Done means the fourth Python RNG draw advances correctly, outputs match eager execution, and the RNG states match, with a regression test covering the reproducer.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.