SpirV inline linking and optimization complexity explosion

Before I start: I'm not sure how much of this is actionable, but @Firestar99 suggested I summarize my recent findings in an issue so here I am.

I have recently been attempting to implement a set of compute shaders for a fairly complex algorithm. I finally got it to the point where I could attempt to compile it with spirv-builder and found that the initial successful compile took on the order of 30 minutes. Discussion with Firestar led me to flags for outputting the total timings of various parts of the compilation, finding the majority of the build time took place in the following steps:

• link_inline (about a minute)
• link_block_ordering_pass_and_mem2reg-after-inlining (about 3 minutes)
• link_spirv_opt (about 31 minutes)

This lead to discussion about how any functions that touch GPU buffers will be forced inline and optimized, which seems a fairly important thing to note. So, I started digging through my functions looking for things that don't make any sense with the knowledge that they would be inlined and here are the things that I found:

The Big Inline

let result = if condition {
    big_function(args..., mut_buffer_a, mut_buffer_b, more_args...)
} else {
    big_function(args..., mut_buffer_b, mut_buffer_a, more_args...)
}

Needing to access these swapping buffers before and after the function call limited my options. I couldn't just hand off the references to a set of first/second buffers because the code later wouldn't have any understanding of which one was which unless i also propagated the condition along with it. My solution here was to move the conditional inside of big_function and move them to a new set of variables in the proper order before anything else uses them. Doing this cut down the lengthy build calls by about 2/3rds resulting in a total compile time in the order of 10 minutes.

The Big Switch

match case {
    1 => {
        func(args1..., buffers...);
        func(args2..., buffers...);
        func(args3..., buffers...);
        func(args4..., buffers...);
        func(args5..., buffers...);
    },
    2 => {
        func(args6..., buffers...);
        func(args7..., buffers...);
        func(args8..., buffers...);
        func(args9..., buffers...);
    },
    // etc
}

This one I was debating about cleaning up before for other reasons, but it turns out doing it this way is a nightmare for the forced inlines too. What was essentially happening here is a 52 case switch where each case calls the same function either 4 or 5 times in a row with different set of arguments. The solution was pretty simple, instead of calling the function in the switch, use the switch to solve for the argument set and call the functions as applicable afterward. Changing this cut the compile time down to a nice 10 seconds.

However, attempting to solve this the first way I tried gave me another issue in the builder...

Some(tuple)

Since the cases in the switch above needed an inconsistent number of function calls, I had at first attempted to use an optional tuple for the arguments of the calls which may not happen in a given case:

let (first, second, third, fourth, fifth) = match case {
    1 => (args1, args2, args3, args4, Some(args5),
    2 => (args6, args7, args8, args9, None,
    // etc
}
func(first..., buffers...);
func(second..., buffers...);
func(third..., buffers...);
func(fourth..., buffers...);
if let Some(fifth) = fifth {
    func(fifth..., buffers...);
}

When compiling this, the compiler gave me a huge bunch of errors regarding trying to convert the optional tuple (in my case a set of u32s) back or forth to a slice of u8s. I didn't linger too much on this since it was more important to me to solve the switch inline problem, but it was pretty clear that the Optional was trying to do some buffer conversion. I don't know if this is supposed to happen or not, but it stuck out enough for me to think I ought to bring it up here.

Start with spirv-builder's timing output and reproduce the shader patterns described in the issue. Investigate the link_inline, link_block_ordering_pass_and_mem2reg-after-inlining, and link_spirv_opt stages, then check the reported Optional buffer-conversion errors. Done means identifying a confirmed compiler defect or documenting the limits and cause of the optimization-time explosion.