KhronosGroup / KhronosGroup/Vulkan-ValidationLayers
Missing GPU validation for VkAccelerationStructureInstanceKHR.accelerationStructureReference in vkCmdBuildAccelerationStructuresKHR
@arno-lunarg is already working on this.
Since Mar 14, 2025.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 505
- Avg merge
- 11h 25m
- Merged PRs (30d)
- 229
Description
I just spent a couple of days hunting down a device lost error and thought I'd share some details. The short version is that validation (as far as I know) does not do GPU validation in vkCmdBuildAccelerationStructuresKHR for the BDAs in VkAccelerationStructureInstanceKHR.accelerationStructureReference. Could we add that to GPU-AV?
Longer version:
I threw a lot of tools at this bug, I of course started with validation/sync val (I'm having problems with GPU-AV right now which have been separately reported) and it ran clean. I also tried the NVIDIA Raytracing Validation (VK_NV_ray_tracing_validation) and it did not report anything. I used Aftermath, Radeon GPU Detective, and CDL to analyze the device lost. It was pretty clear from Aftermath and RGD that the problem was a TLAS build that was referencing a deleted buffer. They both did a good job of identifying that.
I spent a long time focused on the fact that I must be deleting the buffer associated with the BLAS (VkAccelerationStructureCreateInfoKHR.buffer) prior to GPU execution of the vkCmdBuildAccelerationStructuresKHR. That wasn't really what was going on, but it took me a long time to realize it because of how subtle the bug was. The problem was:
- In our code we would queue up the request to a thread to do the TLAS build
- The TLAS build takes a count for
VkAccelerationStructureBuildRangeInfoKHR.primitiveCount - The value being written for the
VkAccelerationStructureBuildRangeInfoKHR.primitiveCountwas being modified in the next frame if the queued command executed before the next frame's TLAS build. If the TLAS had grown, it would read a count larger than the current count. - The GPU buffer holding the VkAccelerationStructureInstanceKHR.accelerationStructureReference was persistent so it would read BDAs from a previous TLAS build that now had deleted buffers (the BDAs at instanceBuffer[count( frame N-1 )...count( frame N )].
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.