IOCP-style queue interface is more appropriate

REVISION (2017-27-05)

After some consideration, and that I can't seem to find the other post right now, I believe an IOCP-styled interface is more efficient, not to mention threaded-application-friendly. This will require some fundamental changes, and two new entry points.

VkResult vkQueueCompletionSubmit(
    VkQueue queue,
    uint32_t submitPtrCount,
    VkSubmitInfo **ppSubmits // Array of pointers - each pops out one-at-a-time, or in batches
    );

VkResult vkQueueCompletionWait(
    VkQueue queue,
    uint32_t *pSubmitPtrCount,    // input is array size, output is popped size
    VkSubmitInfo **ppSubmits,    // no guarantee on order, and one not needed!
    uint64_t timeout
    );

This allows the application to provide pointers to VkSubmitInfo structures, possibly (most likely) wrapped inside other structures, to the driver without having to sort and sift many different arrays of handles.

Implications here include that all the VkSubmitInfo structures, and everything they point to, must remain valid during their trip through the driver.

The idea doesn't specifically require VkSubmitInfo, and perhaps an API-specific structure would be more appropriate. This was just a quick sketch to illustrate the idea.

ORIGINAL (Titled "Strided APIs" - a.k.a wonky select-style use of fences)

I find myself doing a lot of this:

std::vector<VkCommandPool> returnPools;       // allocation hazard and associated overhead
std::vector<VkCommandBuffer> returnBuffers;   // allocation hazard and associated overhead
std::vector<int> mapping;                     // allocation hazard and associated overhead

// ... big fugly select loop over fence handles ...

// sort buffers by their destination pool
assert(returnPools.size() == returnBuffers.size());
mapping.resize(returnPools.size());
auto itr = mapping.begin();
const auto end = mapping.end();
uint32_t value = 0;
while(itr != end){ *itr++ = value++; }
itr = mapping.begin();
std::sort(itr, end, [&](auto a, auto b)
{ return returnPools[a] < returnPools[b]; });

std::vector<VkCommandPool> sortedPools;
std::vector<VkCommandBuffer>sortedBuffers;
for(auto index  : mapping){
    sortedPools.push_back(returnPools[index]);
    sortedBuffers.push_back(returnBuffers[index]);
}

auto poolItr = sortedPools.begin();
auto poolEnd = sortedPools.end();
auto bfrItr = sortedBuffers.begin();
while(poolItr != poolEnd){
    auto pool = *poolItr;
    auto itr0 = poolItr;
    uint32_t count = 1;
    while(++poolItr != poolEnd && pool == *poolItr){ ++count; }

    // ...omitting pool sync. code

    vkFreeCommandBuffers(device, pool, count, &*bfrItr);
    bfrItr += count;
}

When it would be nice to do this:

struct BufferReturn
{
    VkCommandPool pool;
    VkCommandBuffer buffer;
};

std::vector<BufferReturn> returns;

// ... very long code here ...

auto itr = returns.begin();
const auto end = returns.end();
std::sort(itr, end, [&](auto &a, auto &b)
{ return a.pool < b.pool; });

while(itr != end){
    auto pool = itr->pool;
    auto itr0 = itr;
    while(++itr != end && pool == itr->pool){}
    // ...also omitting pool sync. code
    vkFreeCommandBuffersStrided( device, pool,
        (uint32_t)(itr - itr0),
        &itr0->buffer,
        sizeof(BufferReturn) );
}

Where:

vkFreeCommandBuffersStrided(
    VkDevice device,
    VkCommandPool pool,
    uint32_t bufferCount,
    VkCommandBuffer *data,
    size_t stride ); // <- offset, in bytes, between successive buffer handles

TL;DR

In order to minimize API calls, it is best to use them in vectored form. That way, even if an API is slow, the number of invocations is minimized.

Unfortunately, they all assume we can just allocate big homogeneous arrays of handles on the spot.

We will most likely end up putting handles inside structures that get passed around instead of the handles themselves, which could also include the data needed for proper synchronization.

Strided APIs would help immensely by reducing the number of distinct collections that need to be managed solely for the purpose of interfacing with the driver.