Lightning-AI / Lightning-AI/lightning-thunder
ThunderFX failure: KeyError: 'l_stack0_'
@kshitij12345 is already working on this.
Since Oct 14, 2024.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
On the latest Thunder (dafc79d21c04769c5e9d1fb737b8cd21d0841e69) I tried the snippet from
https://github.com/Lightning-AI/lightning-thunder/issues/1174#issuecomment-2383823134
and got the following error:
```py
File ~/dev/lightning-thunder/thunder/dynamo/compiler.py:81, in ThunderCompiler.__call__(self, gm, sample_args)
77 recompile_graph(gm)
79 # The whole graph may not be supported by `thunder`, so we split it in `thunder` supported sections
80 # and unsupported sections which are passed to `torch.compile(backend='inductor')`
---> 81 split_module, subgraph_info = _splitter(gm, self._thunder_jit, self._torch_compile, sample_args)
82 self.subgraph_infos.append(subgraph_info)
83 return split_module
File ~/dev/lightning-thunder/thunder/dynamo/splitter.py:133, in _splitter(gm, thunder_jit, torch_inductor, _unused_sample_args)
130 return partition_cnt
132 # `split_module` iterates over nodes and determines the partition to place them based on the callback.
--> 133 split_gm: torch.fx.GraphModule = split_module(
134 gm, root_m=None, split_callback=callback, keep_original_order=True, keep_original_node_name=True
135 )
137 def is_thunder_supported_partition(node: torch.fx.Node) -> bool:
138 return node.name.startswith("submod") and int(node.name.replace("submod_", "")) in supported_partitions
File ~/dev/pytorch/main/torch/fx/passes/split_module.py:513, in split_module..(n)
510 for node in m.graph.nodes:
511 if node.op == "output":
512 base_mod_graph.output(
--> 513 torch.fx.graph.map_arg(node.args[0], lambda n: base_mod_env[n.name])
514 ) # noqa: B950
516 return torch.fx.graph_module.GraphModule(base_mod_attrs, base_mod_graph)
BackendCompilerFailed: backend='' raised:
KeyError: 'l_stack0_'
```
My PyTorch version is `'2.4.0a0+git3827810'`.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.