Frontend performance

We are trying to improve performance of our application that uses Accelerate on CUDA backend. Recently we came to the realization, that frontend phase of Accelerate is slowing us down. That's because of our use case - under the scenes, user builds a computation, that has to be compiled, computed, and the result viewed on the screen. In other words, latency (time between requesting the result to receiving it) is more important than throughput (performance of the generated kernels). Using run1 family of functions is not feasible, as it would cause a major redesign of our app, and most of the computations are only run once.

I prepared a simple test program that uses run exactly once and generates ~~three~~ two CUDA kernels. I cannot share the source code nor the binary with you.
When I run the program with +RTS -s -p options, it completes in about 2.5s.
Profiling reveals, that convertAccWith function is responsible for 94% of inherited time. Whole .prof file is available here: https://gist.github.com/mikusp/292f47c39847ffb98d79

I thought that disabling optimizations in accelerate by passing flags -fno-simplify, -fno-fusion, etc. will speed up things, but I didn't see a measurable difference.

Finally, I tried to optimize the biggest individual offender, that is idxToInt. Patch to accelerate is here: https://gist.github.com/mikusp/46913963c813175cfb3f accelerate-cuda requires two trivial changes to compile after applying this patch. Result is pretty satisfying, because now the test program runs in ~1.8s.

Please let us know if you have any thoughts about frontend performance. Maybe there are some ways to sacrifice kernel performance for compilation speed or other low hanging fruits that would be easy to optimize. I can provide you with more logs/prof files/examples if it'd be helpful.