Lightning-AI / Lightning-AI/lightning-thunder
`vit_hf.py` fails only when NVFuser is enabled
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
*Note*: If you have a model or program that is not supported yet but should be, please use the program coverage template.
## 🐛 Bug
`torch.testing.assert_close(out, thunder_out)` passes successfully without NVFuser. However, with nvfuser, it fails with the following error:
```
Traceback (most recent call last):
File "/teamspace/studios/this_studio/lightning-thunder/examples/quickstart/vit_hf.py", line 38, in
main()
File "/teamspace/studios/this_studio/lightning-thunder/examples/quickstart/vit_hf.py", line 29, in main
torch.testing.assert_close(out, thunder_out)
File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/torch/testing/_comparison.py", line 1519, in assert_close
raise error_metas[0].to_error(msg)
AssertionError: Tensor-likes are not close!
Mismatched elements: 124694 / 128000 (97.4%)
Greatest absolute difference: 0.003966569900512695 at index (93, 851) (up to 1e-05 allowed)
Greatest relative difference: 9.088653564453125 at index (76, 519) (up to 1.3e-06 allowed)
```
### To Reproduce
- To install nvfuser, do
```
pip install nvfuser-cu128-torch27
```
#### Code sample
### Expected behavior
### Environment
- PyTorch Version (e.g., 1.0):
- OS (e.g., Linux):
- How you installed PyTorch (`conda`, `pip`, source):
- Build command you used (if compiling from source):
- Python version:
- CUDA/cuDNN version:
- GPU models and configuration:
- Any other relevant information:
### Additional context
cc. @IvanYashchuk @kevinstephano @csarofeen @jjsjann123 @t-vi
cc @tfogal
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with examples/quickstart/vit_hf.py, especially the torch.testing.assert_close call at line 29, and reproduce it with and without NVFuser after installing nvfuser-cu128-torch27. Compare the two outputs and compiler paths; done means the example's assertion passes when NVFuser is enabled without changing the expected model behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100