Errors in view pipe and system crashes with AMDGPU + ROCM + OpenCL

Is there an existing issue for this?

• I checked and did not find my issue in the already reported ones

Describe the bug

Frequent corruption of image output both in snapshots and center view. Corruptions manifest as black regions, rainbow regions, banding, etc. Typical GPU failure sort of visuals. Only occurs with high quality image processing enabled, but the system crashes occur even with default image processing settings so long as openCL is enabled.

Steps to reproduce

I strongly suspect this is hardware dependent to AMD Radeon GPUs.

1. Probably have an AMD GPU
2. Enable ROCM OpenCL in DarktTable, default processing priority works fine
3. Open any image, I've only tested with .NEF raw files.
4. Go to Color Equalizer
5. Rapidly move up and down any equalizer region, I usually did the second node from the left.
6. Pretty quickly the whole system will lock up as the AMDGPU driver hits some form of an unrecoverable error and crashes, leaving your screen frozen. The computer is still operable behind all that, but the only way to recover graphics is a reboot.
7. (Optional) Enable high quality processing mode and do arbitrary edits, eventually one may corrupt, or your system crashes. This one is harder to reproduce.

Expected behavior

Darktable should not be doing this ideally

Logfile | Screenshot | Screencast

Due to the nature of the crash I don't have a logfile yet, after posting this bug I'll be trying to crash my system again and save the logs (my last attempt failed sadly).

I do, however, have kernel logs. The first section are warning / error messages that occur whenever darktable gets the visual defects (dark sections, etc.), and the crash section is one of the ways my GPU crashes with darktable.

Jul 25 18:52:50 hostname kernel: amdgpu: User Buffer Address: 0x7fa486595000 already allocated by SVM
Jul 25 18:52:52 hostname kernel: amdgpu: User Buffer Address: 0x7fa486595000 already allocated by SVM
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0: [gfxhub] page fault (src_id:0 ring:88 vmid:8 pasid:1934)
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:  Process darktable pid 1703629 thread worker 0 pid 1703699
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:   in page starting at address 0x0000000000000000 from client 10
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0: GCVM_L2_PROTECTION_FAULT_STATUS:0x008012B1
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:          Faulty UTCL2 client ID: SQC (inst) (0x9)
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:          MORE_FAULTS: 0x1
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:          WALKER_ERROR: 0x0
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:          PERMISSION_FAULTS: 0xb
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:          MAPPING_ERROR: 0x0
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:          RW: 0x0
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0: [gfxhub] page fault (src_id:0 ring:88 vmid:8 pasid:1934)
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:  Process darktable pid 1703629 thread worker 0 pid 1703699
Jul 25 18:52:52 hostname kernel: amdgpu 0000:c4:00.0:   in page starting at address 0x0000000000000000 from client 10

Commit

https://github.com/darktable-org/darktable/releases/tag/release-5.6.0

Where did you obtain darktable from?

distro packaging

darktable version

5.6.0

What OS are you using?

Linux

What is the version of your OS?

ArchLinux 7.1.4-zen1-1-zen

Describe your system

• X11
• 64gb RAM
• AMD Radeon RX7700S (Framework 16 laptop version)
• rocm-core version 7.2.4

Are you using OpenCL GPU in darktable?

Yes

If yes, what is the GPU card and driver?

AMD Radeon RX7700S / ROCM driver

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

This is likely a similar issue to #21617 , where I originally posted, but I was told to make a new bug report for it.

So I tried to reproduce the issue further.

Some quick notes:

• The form above says this is in 5.6, but I could reproduce this in 5.4 as well, and I've had the corruption issues since at least 5.x, or more accurately for as long as I've tried on this computer.
• I say above any image should work, to verify that I tested with this play raw by Glen Butcher, and successfully crashed my system with a straight import + color equalizer fiddle: https://discuss.pixls.us/t/playraw-image-that-requires-no-processing/41646
• I was unable to test with the rusticl driver, attempting to turn it on (after installing opencl-mesa) causes darktable to fail compiling basic.cl during opencl init.

There's a tracking bug for a similar AMDGPU crash, and it's entirely possible the crash portion of this issue is just a way to consistently hit that upstream bug. The errors to the image view, however, could be unrelated to the upstream bug. I just, unfortunately, can't really reproduce the visual errors easily anymore as my system crashes frequently now with testing (could just be color equalizer to be fair, I didn't get crashes from other modules, yet at least).

And frankly, I intend to just turn off openCL on this computer, since I've already burned 6+ hours on debugging this, so it's unlikely I'll be able to get reproduction steps for the image corruption parts.

I am interested if anyone can reproduce the crashes though.