Layernorm backward regression in triton 3.4.0 release candidate

Open

#801 opened on Jul 10, 2025 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language: Python
Stars: 6.6k
Forks: 603
Avg merge: 1d 20h
Merged PRs (30d): 47

Description

### 🐛 Describe the bug

FYI: we're seeing a regression in the layernorm backward kernel on shapes 4096 < x <= 8192. See https://github.com/pytorch-labs/tritonbench/issues/264 for details. The regression is caused by register spilling exposed by an LLVM change.

### Reproduce

See repro in https://github.com/pytorch-labs/tritonbench/issues/264

### Versions

```
Environment Report:
-------------------
Operating System: Linux-6.4.3-0_fbk15_hardened_2630_gf27365f948db-x86_64-with-glibc2.34
Python version: 3.12.11
Liger Kernel version: 0.5.10
PyTorch version: 2.9.0a0+gitc13a346
CUDA version: 12.6
HIP(ROCm) version: Not available
Triton version: 3.4.0
Transformers version: 4.46.1
XPU version: XPU Not Available
```

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Research direction

Start with the reproduction linked from tritonbench issue 264 and inspect the layernorm backward kernel behavior for shapes 4096 < x <= 8192 under Triton 3.4.0. Compare the affected results with the prior release and verify that the register-spilling regression is resolved by rerunning the linked reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack: python
Domain: machine-learning, performance
Issue type: Bug
Difficulty: 4/5
Estimated time: 3-5 days
Activity status: Stale
Clarity: Mostly clear
Newbie friendliness: 35/100