Race condition in validation of VK_KHR_deferred_host_operations functions

There appears to be a race condition potentially preventing pipeline registration when using vkDeferredOperationJoinKHR and vkGetDeferredOperationResultKHR in a multi-threaded environment following a vkCreateRayTracingPipelinesKHR call. Consider the code example from the extension's documentation page (https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_deferred_host_operations.html):

VkResult result = vkDeferredOperationJoinKHR(device, hOp);
while( result == VK_THREAD_IDLE_KHR )
{
    std::this_thread::yield();
    result = vkDeferredOperationJoinKHR(device, hOp);
}

switch( result )
{
case VK_SUCCESS:
    {
        // deferred operation has finished.  Query its result
        result = vkGetDeferredOperationResultKHR(device, hOp);
    }
    break;

case VK_THREAD_DONE_KHR:
    {
        // deferred operation is being wrapped up by another thread
        //  wait for that thread to finish
        do
        {
            std::this_thread::yield();
            result = vkGetDeferredOperationResultKHR(device, hOp);
        } while( result == VK_NOT_READY );
    }
    break;

default:
    assert(false); // other conditions are illegal.
    break;
}

When multiple threads execute this code on the same operation in parallel, it is possible that inside the call to vkGetDeferredOperationResultKHR, the validator will fail to register the pipeline. More specifically, consider the scenario where one thread gets VK_THREAD_DONE_KHR from vkDeferredOperationJoinKHR, and then enters the spinning vkGetDeferredOperationResultKHR loop. Meanwhile, another thread is processing the last chunk of work in its vkDeferredOperationJoinKHR call. As soon as the work internally completes, the implementation can return VK_SUCCESS on the other thread currently spinning on GetDeferredOperationResultKHR. It is therefore possible (and in my experience very likely) that a thread that received VK_THREAD_DONE_KHR from the Join call will process the if (result == VK_SUCCESS) registration epilogue in DispatchGetDeferredOperationResultKHR before the thread completing the deferred operation in its vkDeferredOperationJoinKHR (and thus ultimately receiving VK_SUCCESS from it) call runs the registration epilogue in DispatchDeferredOperationJoinKHR. This will result in the deferred_operation_pipelines not containing the pipeline yet, and therefore in the cleanup_fn callbacks not being triggered. Ultimately, this results in the next API call trying to use the pipeline failing because it is not seen as created by the validator.

This can lead to the following validation error:
Validation Error: [ VUID-vkGetRayTracingShaderGroupHandlesKHR-pipeline-parameter ] Object 0: handle = 0x2cc2f6ca5e0, type = VK_OBJECT_TYPE_INSTANCE; | MessageID = 0x1f2e8acf | Invalid VkPipeline Object 0x7ce53000000349de. The Vulkan spec states: pipeline must be a valid VkPipeline handle (https://vulkan.lunarg.com/doc/view/1.3.231.1/windows/1.3-extensions/vkspec.html#VUID-vkGetRayTracingShaderGroupHandlesKHR-pipeline-parameter)

We can workaround the issue by ensuring that only the completing thread (receiving VK_SUCCESS from vkDeferredOperationJoinKHR) calls vkGetDeferredOperationResultKHR, but it appears like this shouldn't be required.

Environment:

• OS: Windows 10 22H2
• GPU: GeForce 3080
• SDK or header version if building from repo: SDK 1.3.231