KhronosGroup / KhronosGroup/Vulkan-Loader
"Invalid physicalDevice" error when using OpenXR within a Vulkan Layer
- Dominant language
- C
- Stars
- 695
- Forks
- 343
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 17
Description
**Describe the bug**
When trying to use OpenXR's `XR_KHR_vulkan_enable` and `XR_KHR_vulkan_enable2` extensions in a Vulkan layer, it's possible for the OpenXR runtime (code outside of the layer's control) to call `vkPhysicalDevice*` functions that came from the wrong `vkGetInstanceProcAddress` using the wrapped `VkPhysicalDevice` handle which the layer has access to. This causes an `Invalid physicalDevice` error from the loader.
**Environment (please complete the following information):**
- OS: Arch Linux
- Bitdepth: 64-bit
- GPU: NVIDIA GeForce RTX 4070 Ti SUPER
- Graphics Driver: Nvidia 590.48.01 (open-source kernel modules)
- Enabled layers: VK_LAYER_VALVE_steam_overlay_64, my own layer.
- OpenXR Runtime: Monado (commit 7ff112faf7ea0104a78e9389f1a942716d71af18)
**To Reproduce**
I initialize an OpenXR instance from inside my layer by:
1. Hooking vkCreateInstance and vkCreateDevice
2. Within vkCreateInstance, I call:
2a. xrCreateInstance
2b. xrGetSystem
2c. xrEnumerateEnvironmentBlendModes
2d. xrGetVulkanGraphicsRequirementsKHR
2e. xrCreateVulkanInstanceKHR (a wrapper around vkCreateInstance which ensures that the OpenXR runtime's requirements are met. I am careful to prevent the OpenXR runtime from calling into my hook)
3. Within vkCreateDevice
3a. xrGetVulkanGraphicsDevice2KHR
3b. xrGetVulkanDeviceExtensionsKHR
3c. xrCreateVulkanDeviceKHR (another wrapper, similar situation to 2e)
3d. xrCreateSession
3e. xrCreateReferenceSpace
3f. xrEnumerateViewConfigurationViews
3g. xrCreateSwapchain
3h. xrEnumerateSwapchainImages
**VK_LOADER_DEBUG output**
[loader_debug_output.log](https://github.com/user-attachments/files/26107023/loader_debug_output.log)
**Additional context**
Monado calls `vkGetPhysicalDeviceMemoryProperties` [here](https://gitlab.freedesktop.org/monado/monado/-/blob/main/src/xrt/auxiliary/vk/vk_bundle_init.c#L1210), which is what triggers the issue for me specifically. It got that function pointer from [here](https://gitlab.freedesktop.org/monado/monado/-/blob/main/src/xrt/state_trackers/oxr/oxr_session_gfx_vk.c#L146), seemingly just from wherever they initially loaded it.
[This project](https://github.com/Crementif/BotW-BetterVR/blob/main/src/hooking/layer.cpp) gets around this issue by creating the OpenXR instance as a d3d12 instance and using d3d12-Vulkan interop to blit Vulkan images to a d3d12 swapchain; however, this approach is not viable for my use case, as I would like to keep things cross-platform.
Contributor guide
Research direction
Start with the linked VK_LOADER_DEBUG output and the reproduction sequence, then trace the loader entry points involving vkGetPhysicalDeviceMemoryProperties and vkGetInstanceProcAddr. Done should mean identifying a supported handling for the wrapped VkPhysicalDevice across the OpenXR Vulkan-layer path and demonstrating that the Invalid physicalDevice error no longer occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- computer-graphics
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100