Lightning-AI / Lightning-AI/lightning-thunder
Run `examine` automatically when hit unsupported operator failure
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
`examine` is a very useful and clean method that analyzes the network and reports the unsupported operators. When Thunder hits an unsupported operator, it would be very useful to have Thunder automatically run `examine` internally with the model and the inputs it already has and report the `examine` output along with the error.
### Motivation
As Thunder grows in adoption, we are seeing it being used on models with operators which are not currently supported. Today, Thunder throws an error that's not very clear that the error is attributed to an unsupported operator.
Example, see comment from a contributor below -- https://github.com/Lightning-AI/lightning-thunder/issues/93#issuecomment-2037641430
Another motivation for this comes from more complex libraries like NeMo where the model builder and the dataloader are not often in the same piece of code and is not easily accessible as they use PyTorch Lightning Trainer etc. While there is an option to create dummy inputs to run `examine`, this creates a layer of friction between adoption. Often, the engineer running the model is not the same as the one who built the model and may not even have the understanding of the input data expected by the model.
CC - @mruberry @tfogal
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing where Thunder reports unsupported-operator failures and how the existing `examine` entry point receives a model and its inputs. Done means those failures include the `examine` output generated from the model and inputs already available to Thunder.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100