ROCm / ROCm/amdgpu

NULL pointer dereference in amdgpu_ttm_tt_unpopulate() panics the kernel

Open
#222 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
460
Forks
143
PR merge metrics
No merged PRs in 30d

Description

Summary

amdgpu_ttm_tt_unpopulate() dereferences ttm->pages[i] without a NULL check.
When a GPU command fails partway through, the buffer object can be left
partially populated, and the cleanup path then panics the kernel.

On an AMD BC-250 (gfx1013 / Cyan Skillfish, PS5-derived APU) running ROCm
compute, this turned every GPU fault into a hard machine hang — four in one
debugging session, with unclean shutdowns and filesystem data loss.

Trace

RIP: 0010:amdgpu_ttm_tt_unpopulate+0x77/0xd0 [amdgpu]
CR2: 0000000000000018
RAX: 0000000000000000  RBX: ffff8c72d0b6e180  RCX: 0000000000000000
RDX: 00000000ffffffff  RSI: 0000000000000000  RDI: ffff8c729f580000
Kernel panic - not syncing: Fatal exception

The faulting instruction is the store inside the loop:

48 8b 0b                  mov  (%rbx), %rcx           ; rcx = ttm->pages
48 8b 0c c1               mov  (%rcx,%rax,8), %rcx    ; rcx = pages[i]
48 c7 41 10 00 00 00 00   movq $0, 0x10(%rcx)         <- panic

RCX = 0 and CR2 = 0x18 (the offset of struct page::mapping) confirm
pages[i] was NULL.

Code

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c:

	for (i = 0; i < ttm->num_pages; ++i)
		ttm->pages[i]->mapping = NULL;

Suggested change

	for (i = 0; i < ttm->num_pages; ++i)
		if (ttm->pages[i])
			ttm->pages[i]->mapping = NULL;

Caveat

This addresses the crash, not the cause. Why a buffer object ends up
partially populated on this hardware is a separate problem I have not solved —
the board has a compute bug where the second heavy GPU phase in a process
reliably fails, and I have not root-caused it.

I am reporting this because unpopulate arguably should not dereference a NULL
page regardless of how it got there, and because the difference between "the
process dies" and "the machine dies" is significant when debugging.

If the correct fix is upstream of this loop, that's a better outcome — I just
don't know where.

Environment

board    AMD BC-250, gfx1013:xnack- (1002:13fe), 40 CU
kernel   7.0.12 (CachyOS, with BC-250 patches)
ROCm     7.2.4

Testing

Regression tested with a full Stable Diffusion 1.5 pipeline (512x512, 24 steps).
Output byte-size identical before and after; timings unchanged:

module 1st run 2nd run
unpatched 54.33s / 40.79s 33.23s / 31.36s
patched 42.22s 33.16s

A later run that produced 2 page faults left the machine up, with
Kernel panic: 0. Two faults is an indication, not proof — but it is the first
time in that session the machine survived them.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c at amdgpu_ttm_tt_unpopulate(), then inspect the population and cleanup paths that can leave ttm->pages partially populated. Verify the fix against the reported partial-page case and run the Stable Diffusion 1.5 pipeline; done means GPU faults no longer panic or hang the kernel.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
operating-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.