llvm / llvm/llvm-project

[B200] LLVM NVPTX mixed-precision folding causes register and occupancy regression

Open
#213,332 0 comments 0 reactions 0 assignees View on GitHub
backend:NVPTX
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

LLVM’s NVPTX backend unconditionally folds multi-use FP16-to-FP32 extensions into mixed-precision instructions on sm_100. Combined with --nvptx-mad-wide-opt, this can increase register usage from 64 to 72 registers/thread, reducing 512-thread block residency from two blocks per SM to one.

This LLVM issue manifests in Triton-generated kernels, including a layer-norm kernel that regresses by approximately 50% on B200.

- Triton reproducer and investigation: https://github.com/triton-lang/triton/issues/11127
- Proposed LLVM fix by coding agent: https://github.com/llvm/llvm-project/pull/213327

The proposed LLVM change restores 64 registers and reduces latency from 84.75 µs to 53.45 µs in the standalone B200 reproducer. https://gist.github.com/yushangdi/6ca5cf780ccd85ac1586f2b77bdf00a1

Contributor guide

Open the contributing guide

Research direction

Start with the Triton reproducer and investigation in issue 11127, then read the proposed LLVM change in pull request 213327 alongside the standalone B200 reproducer. Done means avoiding the register increase from 64 to 72 registers per thread and confirming the reported occupancy and latency improvements.

Written by the indexing model from the issue text.

Assessment

Domain
compilers, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.