obsproject / obsproject/obs-studio

Unvalidated parameters in Vulkan capture layer

Open
#6,149 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
76.4k
Forks
10.2k
Avg merge
4d 23h
Merged PRs (30d)
12

Description

Operating System Info

Windows 10

OBS Studio Version

27.2.3

OBS Studio Log URL

N/A

Expected Behavior

When calling vkCreateSwapchainKHR in a Vulkan application the spec states: The implied image creation parameters of the swapchain must be supported as reported by vkGetPhysicalDeviceImageFormatProperties. OBS should respect the spec and check if its changes to the structure are valid.

Current Behavior

Current code: https://github.com/obsproject/obs-studio/blob/02e54103813f90eac34530de95c868251df01b9b/plugins/win-capture/graphics-hook/vulkan-capture.c#L1659-L1666

It just tries vkCreateSwapchainKHR with a modified imageUsage and if it fails it falls back to the original parameters.

Steps to Reproduce

I have been following this tutorial: https://vulkan-tutorial.com/Introduction
At the swapchain creation step: https://vulkan-tutorial.com/Drawing_a_triangle/Presentation/Swap_chain

I had the Khronos validation layer enabled which then showed this error message:

VUID-VkSwapchainCreateInfoKHR-imageFormat-01778(ERROR / SPEC): msgNum: -1070202321 - Validation Error: [ VUID-VkSwapchai
nCreateInfoKHR-imageFormat-01778 ] Object 0: handle = 0x222d7f72230, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xc0360
22f | vkCreateSwapchainKHR(): pCreateInfo->imageFormat VK_FORMAT_B8G8R8A8_SRGB with tiling VK_IMAGE_TILING_OPTIMAL has n
o supported format features on this physical device. The Vulkan spec states: The implied image creation parameters of th
e swapchain must be supported as reported by vkGetPhysicalDeviceImageFormatProperties (https://vulkan.lunarg.com/doc/vie
w/1.3.204.1/windows/1.3-extensions/vkspec.html#VUID-VkSwapchainCreateInfoKHR-imageFormat-01778)
    Objects: 1
        [0] 0x222d7f72230, type: 3, name: NULL
Anything else we should know?

GPU: AMD RX 580
Driver Version: 22.2.3

Using the Vulkan SDK provided here: https://vulkan.lunarg.com/sdk/home
Version: 1.3.204.1

I tried disabling the layer using the Vulkan Configurator provided with the SDK which fixes the validation error.
On that note: The Vulkan layer is implicitly on even if OBS is not running, it just has to be installed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with plugins/win-capture/graphics-hook/vulkan-capture.c around lines 1659-1666 and trace the modified vkCreateSwapchainKHR call. Compare the current fallback behavior with vkGetPhysicalDeviceImageFormatProperties and the Vulkan validation error from the reproduction. Done means the capture layer no longer submits unsupported modified swapchain parameters.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.