llvm / llvm/llvm-project

[AMDGPU] Illegal code generated by RewriteMFMAForm stage

Open
#214,124 7 comments 0 reactions 1 assignee Claimed by @xgxanq View on GitHub
backend:AMDGPU
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

RewriteMFMAForm stage can produce illegal code: the current logic that checks if we can reclassify register as AGPR is not sufficient to handle all the cases. In the example below it changes ds_read's operand to AGPR type, which is not supported - we should check all the uses (and (re)defs) when changing register's class.

```
# RewriteMFMAForm produces machine-verifier-illegal code:
#
# Run:
# llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx950 -run-pass=machine-scheduler \
# -amdgpu-sched-strategy=coexec -verify-machineinstrs mfma-agpr-verifier-bug.mir \
# -o /dev/null
#
# *** Bad machine code: Illegal virtual register for instruction ***
# - function: mfma_agpr_verifier_bug
# - basic block: %bb.0 (0x55e5795b1218) [0B;288B)
# - instruction: 184B %ld:vgpr_32 = DS_READ_B32_gfx9 %acc.sub0:areg_64_align2, 0, 0, implicit $exec :: (load (s32), addrspace 3)
# - operand 1: %acc.sub0:areg_64_align2
# AReg_64_Align2.sub0 cannot be used for VGPR_32 operands.LLVM ERROR: Found 1 machine code errors.

--- |
define amdgpu_kernel void @mfma_agpr_verifier_bug() #0 { ret void }
attributes #0 = { "amdgpu-flat-work-group-size"="1,256" }
...
---
name: mfma_agpr_verifier_bug
tracksRegLiveness: true
isSSA: false
body: |
bb.0:
liveins: $vgpr0_vgpr1, $vgpr2_vgpr3

%s0:vreg_64_align2 = COPY $vgpr0_vgpr1
%s1:vreg_64_align2 = COPY $vgpr2_vgpr3

; inflate ArchVGPR pressure > 256 (9 x vreg_1024 = 288) so RewriteMFMAForm runs
%p0:vreg_1024 = IMPLICIT_DEF
%p1:vreg_1024 = IMPLICIT_DEF
%p2:vreg_1024 = IMPLICIT_DEF
%p3:vreg_1024 = IMPLICIT_DEF
%p4:vreg_1024 = IMPLICIT_DEF
%p5:vreg_1024 = IMPLICIT_DEF
%p6:vreg_1024 = IMPLICIT_DEF
%p7:vreg_1024 = IMPLICIT_DEF
%p8:vreg_1024 = IMPLICIT_DEF

%acc:vreg_64_align2 = AV_MOV_B64_IMM_PSEUDO 0, implicit $exec
dead %mdst:vreg_64_align2 = V_MFMA_F64_4X4X4F64_vgprcd_e64 %s0, %s1, %acc, 0, 0, 0, implicit $mode, implicit $exec

; redef after the MFMA -> new whole-reg value number
%acc.sub0 = AV_MOV_B32_IMM_PSEUDO 7, implicit $exec

%ld:vgpr_32 = DS_READ_B32_gfx9 %acc.sub0, 0, 0, implicit $exec :: (load (s32), addrspace 3)

dead %conn:vreg_64_align2 = COPY %acc
S_ENDPGM 0, implicit %ld, implicit %p0, implicit %p1, implicit %p2, implicit %p3, implicit %p4, implicit %p5, implicit %p6, implicit %p7, implicit %p8
...

```

tested on llvm:main at
commit 2ff5099be4ea2f88609c23037fdbe4e355739a04 (HEAD -> amdgpu_rwmfma_stage_fix, upstream_fork/main)
Author: Florian Hahn
Date: Mon Aug 3 19:34:29 2026 +0200

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.