Do not auto select NVTE FUSED if CUDNN_FRONTEND version is less than 1.15.0
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 272
Description
**Describe the bug**
A clear and concise description of what the bug is.
The heuristic for selecting NVTE-FUSED is aggressive and can accidentally trigger non-deterministic NaNs on the SDPA attention when dispatched via CUDNN frontend (fixed in most recently release). Should guarand against all but the newer transformer engine releases. Latest PyTorch bug discussion is here: https://github.com/pytorch/pytorch/issues/166211
**Steps/Code to reproduce bug**
Please list *minimal* steps or code snippet for us to be able to reproduce the bug.
A helpful guide on on how to craft a minimal bug report http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
Contributor guide
Research direction
No files or tests are named. Start by locating the heuristic that selects NVTE-FUSED and how it detects the CUDNN frontend version; confirm the relevant Transformer Engine or PyTorch integration path. Done means NVTE-FUSED is not automatically selected when CUDNN_FRONTEND is older than 1.15.0, while newer supported versions retain the intended behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100