KhronosGroup / KhronosGroup/Vulkan-ValidationLayers

Object wrapping and debug utils callbacks

Open
#6,972 0 comments 0 reactions 0 assignees View on GitHub
Bug
Dominant language
C++
Stars
1k
Forks
504
Avg merge
11h 35m
Merged PRs (30d)
224

Description

**Describe the Issue**

The unique handle wrapping feature results in VVL replacing object handles returned by layers/ICD with VVL's own handles. VVL then handles the wrapping/unwrapping to ensure that this is all hidden from the application.

However, it does not perform this translation on handles passed to the debug utils callback via `VkDebugUtilsMessengerCallbackDataEXT::pObjects[0]::objectHandle`. A specific case where this is visible is when using `VK_EXT_device_address_binding_report`, which uses the debug callbacks to return memory binding messages.

I believe the sequence of events here is:

1. Application creates a debug messenger, and passes a callback function pointer
2. VVL sees that the ICD implements `VK_EXT_debug_utils` (which is a required dependency of `VK_EXT_device_address_binding_report`) and so defers to the ICD to create the debug messenger
3. The application calls an object creation function and VVL returns a wrapped handle
4. The ICD fires a memory binding message via the debug callback, but passes it's own handles in `VkDebugUtilsMessengerCallbackDataEXT::pObjects[0]::objectHandle`. The debug callback that gets invoked is the applications own function pointer, and so VVL doesn't get to intervene/wrap the handles to match those returned by step 3. The application now has no way to correlate the two handles.

It seems like VVL should be able to intercept the creation of the debug messenger, and pass it's own version of the debug messenger callback down to the ICD/lower layers. VVL's callback would need to perform wrapping/unwrapping and then invoke the applications own callback.

I haven't actually tried to implement this, so there may well be corner cases that make this not practical. If that's the case, then the fallback solution would likely be to add a note to the spec/extension documentation recommending disabling object wrapping in VVL when using `VK_EXT_device_address_binding_report`. However, since both the extension and VVL are useful debugging tools, it would be preferable to find a route to making them work together.

**Expected behavior**

From the applications PoV: handles returned via the debug callbacks should be consistent with those returned via the object creation functions.

**Additional context**

Whilst the motivating use case for this is `VK_EXT_device_address_binding_report`, I *think* you can trigger the same behaviour by having a 2nd layer located underneath VVL, which then calls `vkSubmitDebugUtilsMessageEXT` after the application has created a debug messenger.

Contributor guide

Open the contributing guide

Research direction

Start at debug messenger creation and the callback path, including vkSubmitDebugUtilsMessageEXT and the object-wrapping logic described in the issue. Determine how a validation-layer callback could translate pObjects objectHandle values before invoking the application's callback; done means callback handles consistently match handles returned by object-creation functions, including device address binding reports.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.