Lightning-AI / Lightning-AI/lightning-thunder

Pattern Matching

Open
#2,561 1 comment 0 reactions 1 assignee View on GitHub

@riccardofelluga is already working on this.

Since Oct 24, 2025.

developer efficiency high priority pattern matching
Dominant language
Python
Stars
1.5k
Forks
121
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature

Match and replace computation, in particular sequences of multiple operations. This feature will make it easier and faster for developers to improve model performance of new models by replacing computation. Therefore resulting in faster delivery of optimizations.

### Motivation

At the moment in Thunder it is possible to do limited computation pattern replacements by specifying an executor that supports a limited set of operations, of which all are the ones we want to replace (claim by the executor, [see xentropy](https://github.com/Lightning-AI/lightning-thunder/blob/cc09df9045776899102c4dcd3df55feebe89059f/thunder/executors/torch_compile.py#L236-L274)). This is limited functionality for three reasons:
1. It is really not nice to ask the user to specify a new executor and declare the symbols they want to replace
2. Using the executor to claim the operation it is not possible to replace the pattern with more than one symbol. I.e. In the trace the pattern will be substituted by one bound symbol which will be used to call the implementation specified in the executor that the user just created.
3. Even if it would have been possible to replace with more than one symbol, the pattern specification through executor lacks order information, and therefore does not always match to the same chain of ops.
4. *[NEW]* custom op does not allow to specify multiple operations to be replaced.

cc @lantiga @borda @mruberry

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.