pytorch / pytorch/pytorch

AOTI infinite recursion from a scalar buffer

Open
#169,469 6 comments 0 reactions 1 assignee Claimed by @desertfire View on GitHub
export-triaged module: aotinductor oncall: export oncall: pt2 triaged
Dominant language
Python
Stars
103k
Forks
29.6k
PR merge metrics
PR metrics pending

Description

### 🐛 Describe the bug

Unfortunately I cannot seem to create a self contained MRE, and I am unable to share my code. However the error I was getting from aoti_compile_and_packaging my exported program was the following.

```
args = (tensor(..., device='meta', size=(4,), dtype=torch.uint8), torch.uint8), kwargs = {} def fake_impl(*args, **kwargs): if self._abstract_fn is None: if utils.can_generate_trivial_fake_impl(self._opoverload): return None raise RuntimeError( f"There was no fake impl registered for {self}. " f"This is necessary for torch.compile/export/fx tracing to work. " f"Please use {self._init_fn.__name__}.register_fake to add an " f"fake impl." ) > return self._abstract_fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E RecursionError: maximum recursion depth exceeded
```

when running this through a debugger i saw the recursion started on `constant95` which tracing through the code I saw was a buffer i had in my code which had a name. Eg i had some module with something like

```
self.register_buffer("x", ...) # shape [1, 1]
self.register_buffer("y", ...) # shape [1, 1]
self.register_buffer("z", ...) # shape []
```
but in the graph here https://github.com/pytorch/pytorch/blob/ecbcc3f6bf327856b435b259ac63cc2f328c4b4e/torch/_inductor/compile_fx.py#L1415 my `graph.constants` was something like `[..., "x", "y", "constant95"]` . This led me to instead build my module buffer like
```
self.register_buffer("z", tensor[None]) # shape[1]
```
which solved the problem.

The recursion itself kept going through this function https://github.com/pytorch/pytorch/blob/ecbcc3f6bf327856b435b259ac63cc2f328c4b4e/torch/_decomp/decompositions.py#L2278

also to be clear I did not have any uint8 tensors to my knowledge.

Sorry I cannot share the code or recreate this.

### Versions

torch==2.9.1

cc @chauhang @penguinwu @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan @angelayi @suo @ydwu4 @desertfire @yushangdi @benjaminglass1 @jataylo @iupaikov-amd

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.