[scan] inductor refuses a tanh/sigmoid recurrence under autograd ("scan might be aliasing the input or the output!"): ScanAutogradOp returns the carry twice as a saved intermediate
- Dominant language
- Python
- Stars
- 103k
- Forks
- 29.5k
- PR merge metrics
- PR metrics pending
Description
### 🐛 Describe the bug
`torch.compile(backend="inductor")` refuses any `scan` whose `combine_fn` returns, as its **carry**, a tensor produced by an op that saves its *output* for backward (`tanh`, `sigmoid`, ...), as soon as anything in the scan requires grad:
```
BackendCompilerFailed: backend='inductor' raised:
RuntimeError: scan might be aliasing the input or the output!
```
The `combine_fn` follows the documented contract (`return carry, carry.clone()`; no input mutation, output does not alias input). `aot_eager` compiles and trains it; only the autograd + functionalize path taken under inductor fails. The same body with `requires_grad=False` everywhere compiles under inductor.
This is the canonical RNN spelling, so in practice it means `scan` cannot be used for training a `tanh`/`sigmoid` recurrence under inductor at all.
#### Minimal repro
```python
import torch
from torch._higher_order_ops.scan import scan
torch.manual_seed(0)
B, T, D = 4, 16, 8
W = torch.nn.Parameter(torch.randn(D, D) * 0.1) # requires_grad=True
def rollout(state0, xs):
def body(s, x):
s_new = torch.tanh(s @ W + x)
return s_new, s_new.clone() # documented pattern: emitted slice is a clone of the carry
_, ys = scan(body, state0, xs)
return ys
for backend in ("aot_eager", "inductor"):
torch._dynamo.reset()
try:
ys = torch.compile(rollout, backend=backend, fullgraph=True)(torch.zeros(B, D), torch.randn(T, B, D))
ys.sum().backward()
print(f"{backend:<10} OK W.grad norm = {W.grad.norm():.4f}")
W.grad = None
except Exception as err: # noqa: BLE001
last = [line for line in str(err).splitlines() if "alias" in line][0].strip()
print(f"{backend:<10} FAIL {type(err).__name__}: {last}")
W.requires_grad_(False)
torch._dynamo.reset()
torch.compile(rollout, backend="inductor", fullgraph=True)(torch.zeros(B, D), torch.randn(T, B, D))
print("inductor OK with W.requires_grad=False")
```
Output (identical on 2.13.0+cpu and 2.14.0+cpu):
```
aot_eager OK W.grad norm = 33.0181
inductor FAIL BackendCompilerFailed: RuntimeError: scan might be aliasing the input or the output!
inductor OK with W.requires_grad=False
```
#### What triggers it
Same harness, varying the body and which tensor requires grad (inductor, `fullgraph=True`, 2.14.0):
| body (`s_new = ...`) | grad on | result |
| -- | -- | -- |
| `tanh(s @ W + x)` | `W` | **REFUSED** |
| `tanh(s @ W + x)` | `state0` only (`W` frozen) | **REFUSED** |
| `tanh(s @ W + x)` | nothing | OK |
| `sigmoid(s @ W + x)` | `W` | **REFUSED** |
| `s @ W + x` | `W` | OK |
| `sin(s @ W + x)` | `W` | OK |
| `relu(s @ W + x)` | `W` | OK |
| `tanh(s @ W + x) * 1.0` | `W` | OK |
So it is not about *which* tensor requires grad, and not about closures / `additional_inputs`. It fires exactly when the carry output node is one that autograd saves **as-is** for backward (`tanh_backward` and `sigmoid_backward` take the op's *output*; `mm`/`sin` save their inputs; the `* 1.0` makes the carry a different node from the saved `tanh`).
#### Root cause
Instrumenting `has_potential_input_alias_or_mutation` (called from `scan_functionalize` via `_check_alias_and_mutation`) shows the check is run on the **partitioned forward graph** built by `ScanAutogradOp`, and that graph returns the same node twice:
```
graph():
%primals_0 : [num_users=2] = placeholder[target=primals_0] # carry
%primals_1 : [num_users=1] = placeholder[target=primals_1] # xs slice
%primals_2 : [num_users=2] = placeholder[target=primals_2] # W (additional_input)
%mm = aten.mm(%primals_0, %primals_2)
%add = aten.add(%mm, %primals_1)
%tanh = aten.tanh(%add) # <- the carry
%clone_1 = aten.clone(%tanh) # <- ys (user's clone)
%permute = aten.permute(%primals_0, [1, 0])
%clone_3 = aten.clone(%permute, memory_format=contiguous)
%permute_1 = aten.permute(%primals_2, [1, 0])
%clone_4 = aten.clone(%permute_1, memory_format=contiguous)
return (tanh, clone_1, tanh, clone_3, clone_4)
^^^^ ^^^^
fw output (carry) saved intermediate for tanh_backward: the SAME node
```
`tanh` is both the forward carry output and a saved intermediate, i.e. an output-output alias, which is what the check rejects.
The partitioner has a pass meant to remove exactly this kind of aliasing, but its classification only looks at whether an intermediate **is an input placeholder** (`torch/_higher_order_ops/scan.py`, `ScanAutogradOp`, on `main` today):
```python
for i, out in enumerate(fw_intermediates):
if out in init_node_set: # -> CLONE
elif out in xs_node_set: # -> REMOVE_XS
elif out in addi_node_set: # -> REMOVE_ADDITIONAL_INPUTS
else: # -> KEEP
```
An intermediate that is the same node as one of the **forward outputs** is neither, so it gets `KEEP` and is returned a second time unchanged. The `ScanForwardIntermediatesHandlingPolicy` docstring describes `CLONE` as the treatment for a carried *input*; the carried *output* needs the same treatment when the partitioner saves it.
#### Suggested fix
In the classification above, also clone (or de-duplicate) an intermediate that is one of `fw_outputs`:
```python
fw_output_node_set = set(fw_outputs)
...
elif out in fw_output_node_set:
self.forward_intermediates_handling_policies.append(
ScanForwardIntermediatesHandlingPolicy.CLONE
)
```
(or emit the saved value once and index it in backward). Either way the emitted per-step slice stops aliasing the carry.
#### User-side workaround (verified)
Clone the **carry** as well as the emitted slice, so the saved `tanh` is no longer a forward output:
```python
def body(s, x):
s_new = torch.tanh(s @ W + x)
return s_new.clone(), s_new.clone()
```
| return | result |
| -- | -- |
| `s_new, s_new.clone()` (documented pattern) | REFUSED |
| `s_new.clone(), s_new` | REFUSED |
| `s_new.clone(), s_new.clone()` | OK — forward max\|Δ\| 3e-8, `W.grad` max\|Δ\| 7e-7 vs an eager Python loop |
This costs an extra copy of the carry per step. Note the documented pattern (`return carry, carry.clone()`) is the one that fails, so users following the docs hit this with no indication of what to change; the error text points at their `combine_fn` rather than at the partitioner.
Related but distinct: #191582 / #191686 (input-input aliasing in `while_loop`'s check, from Dynamo lifting a closure that is also a carry). This one is output-output aliasing introduced by `ScanAutogradOp`'s own forward graph.
### Versions
Reproduced on `torch==2.13.0+cpu` and `torch==2.14.0+cpu` (Python 3.14.2, Linux x86_64, WSL2, CPU only). The classification code quoted above is unchanged on `main` as of 2026-09-03.
collect_env (2.14.0 env)
```
Collecting environment information...
PyTorch version: 2.14.0+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.5 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04.3) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35
Python version: 3.14.2 (main, Dec 9 2025, 19:03:28) [Clang 21.1.4 ] (64-bit runtime)
Python platform: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: False
Caching allocator config: N/A
CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 42 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Vendor ID: GenuineIntel
Model name: Intel(R) Core(TM) Ultra 7 258V
CPU family: 6
Model: 189
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
Stepping: 1
BogoMIPS: 6604.80
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni vnmi umip waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize flush_l1d arch_capabilities
```
cc @chauhang @penguinwu @ydwu4 @bdhirsh @bobrenjc93 @aorenste
Contributor guide
Research direction
Start in torch/_higher_order_ops/scan.py at ScanAutogradOp and the fw_intermediates classification described in the issue. Reproduce the minimal tanh recurrence with torch.compile(backend="inductor") and inspect the partitioned forward graph. Done means the saved intermediate no longer aliases the forward carry and the documented recurrence compiles and trains under inductor.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100