KhronosGroup / KhronosGroup/Vulkan-ValidationLayers

Missing GPU validation for VkAccelerationStructureInstanceKHR.accelerationStructureReference in vkCmdBuildAccelerationStructuresKHR

Open
#9,614 5 comments 0 reactions 1 assignee View on GitHub

@arno-lunarg is already working on this.

Since Mar 14, 2025.

GPU-AV Incomplete RT
Dominant language
C++
Stars
1k
Forks
505
Avg merge
11h 25m
Merged PRs (30d)
229

Description

I just spent a couple of days hunting down a device lost error and thought I'd share some details. The short version is that validation (as far as I know) does not do GPU validation in vkCmdBuildAccelerationStructuresKHR for the BDAs in VkAccelerationStructureInstanceKHR.accelerationStructureReference. Could we add that to GPU-AV?

Longer version:
I threw a lot of tools at this bug, I of course started with validation/sync val (I'm having problems with GPU-AV right now which have been separately reported) and it ran clean. I also tried the NVIDIA Raytracing Validation (VK_NV_ray_tracing_validation) and it did not report anything. I used Aftermath, Radeon GPU Detective, and CDL to analyze the device lost. It was pretty clear from Aftermath and RGD that the problem was a TLAS build that was referencing a deleted buffer. They both did a good job of identifying that.

I spent a long time focused on the fact that I must be deleting the buffer associated with the BLAS (VkAccelerationStructureCreateInfoKHR.buffer) prior to GPU execution of the vkCmdBuildAccelerationStructuresKHR. That wasn't really what was going on, but it took me a long time to realize it because of how subtle the bug was. The problem was:

  • In our code we would queue up the request to a thread to do the TLAS build
  • The TLAS build takes a count for VkAccelerationStructureBuildRangeInfoKHR.primitiveCount
  • The value being written for theVkAccelerationStructureBuildRangeInfoKHR.primitiveCountwas being modified in the next frame if the queued command executed before the next frame's TLAS build. If the TLAS had grown, it would read a count larger than the current count.
  • The GPU buffer holding the VkAccelerationStructureInstanceKHR.accelerationStructureReference was persistent so it would read BDAs from a previous TLAS build that now had deleted buffers (the BDAs at instanceBuffer[count( frame N-1 )...count( frame N )].

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.