@nospecialize slows down first time inference

I notice the use of `@nospecialize` slows down the first time inference when using IRTools functions (e.g `Pipe`, `update!`) a lot, not sure why

```jl
# without nospecialize
6.351286 seconds (10.75 M allocations: 572.988 MiB, 5.41% gc time)
# with nospecialize
7.406211 seconds (14.54 M allocations: 773.888 MiB, 5.02% gc time)
```

I suspect this is also causing other things (like Zygote) to be extremely slow for the first run.

Start by reproducing the reported first-inference timings for the IRTools functions Pipe and update!, comparing runs with and without @nospecialize. Trace the relevant inference path and allocations around those entry points. Done means the cause of the slowdown is identified and first-run performance is improved without regressing the affected functions.

Written by the indexing model from the issue text.