KhronosGroup / KhronosGroup/Vulkan-ValidationLayers
Vulkan video decode crashes with unique_handles validation on NVIDIA
- Dominant language
- C++
- Stars
- 1k
- Forks
- 504
- Avg merge
- 11h 35m
- Merged PRs (30d)
- 224
Description
## Summary
With Vulkan SDK 1.4.341.1 validation layers and NVIDIA driver 595.58.03 on Linux, FFmpeg Vulkan H.264 decode segfaults inside libnvidia-glcore.so during vkQueueSubmit2 when VK_LAYER_KHRONOS_validation is enabled.
The crash reproduces with a public FFmpeg sample:
https://samples.ffmpeg.org/V-codecs/h264/missing_frames.avi
The crash requires validation core plus unique handle wrapping:
```text
khronos_validation.validate_core = true
khronos_validation.unique_handles = true
```
The same workload succeeds when `unique_handles=false` while keeping standard validation checks otherwise enabled. No validation message is emitted before the segfault.
## Repro
Download sample:
```bash
curl -L -o missing_frames.avi https://samples.ffmpeg.org/V-codecs/h264/missing_frames.avi
```
Run FFmpeg Vulkan decode with validation enabled:
```bash
VK_LAYER_PATH=/home/aim/vulkan/1.4.341.1/x86_64/share/vulkan/explicit_layer.d \
VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
./ffmpeg -v warning -threads 1 \
-init_hw_device vulkan=vk \
-hwaccel vulkan -hwaccel_output_format vulkan \
-i missing_frames.avi \
-frames:v 8 -f null -
```
Result:
```text
Segmentation fault (core dumped)
exit status 139
```
## Stack excerpt
```text
Thread received signal SIGSEGV
#0 libnvidia-glcore.so.595.58.03
#9 vvl::dispatch::Device::QueueSubmit2
#10 vulkan_layer_chassis::QueueSubmit2
#11 ff_vk_exec_submit
#12 ff_vk_decode_frame
```
## Isolation
Passing case: standard validation settings with only unique_handles disabled:
```text
khronos_validation.validate_core = true
khronos_validation.unique_handles = false
```
Failing minimal combination:
```text
khronos_validation.validate_core = true
khronos_validation.unique_handles = true
```
Other checks tested individually did not reproduce the crash.
## Environment
- Vulkan SDK / validation layers: 1.4.341.1
- NVIDIA driver: 595.58.03
- GPU: NVIDIA GeForce RTX 5080
- OS: Linux (Omarchy)
- Application: FFmpeg master, Vulkan H.264 hardware decode
- Public sample: https://samples.ffmpeg.org/V-codecs/h264/missing_frames.avi
## Notes
This looks like a unique-handle wrapping/unwrapping issue around Vulkan Video objects or submit-time validation. The command stream runs without validation, and it also runs with validation when unique_handles is disabled.
Contributor guide
Research direction
Reproduce the crash with missing_frames.avi using the provided FFmpeg Vulkan command and validation settings. Compare the failing unique_handles=true case with unique_handles=false, then trace the submit path through vvl::dispatch::Device::QueueSubmit2, vulkan_layer_chassis::QueueSubmit2, ff_vk_exec_submit, and ff_vk_decode_frame. Done means the workload no longer segfaults with unique-handle wrapping enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- computer-graphics
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100