Permissibility of Internal Queues in Command Buffers

One nice aspect of command buffers is that they can help facilitate simultaneous execution of kernels. Although this is theoretically possible with out of order queues, it tends to work poorly in practice, with vendor support for such queues being sub-par.

Given this a natural means of implementing command buffers is to scan the resulting graph and use this to see how much parallelism is available. Then, a number of internal (in order) queues can be created and, upon execution, kernels sent to these queues in a prescribed order. (This is how NVIDIA and AMD both implement their graph APIs under the hood with the execution graph having some number of internal streams attached to it. The AMD implementation is open source with https://github.com/ROCm-Developer-Tools/hipamd/blob/develop/src/hip_graph_internal.cpp#L771 being a good starting point.) It is thus important that such a strategy can also be employed with command buffers.

A consequence, however, is that not all of the kernels in the buffer will go to the queue(s) specified in `clEnqueueCommandBufferKHR`. Additionally, it may have some interactions with future multi-queue applications for command buffers.

Start by reading the command-buffer discussion in this issue and the referenced hip_graph_internal.cpp implementation around line 771. Determine whether internal in-order queues are permitted by clEnqueueCommandBufferKHR and how that affects the specified queue and future multi-queue behavior; done requires a resolved specification decision.

Written by the indexing model from the issue text.