Optimization efficacy and efficiency

No one has yet worked on:

• Tuning optimizations to improve performance for real Stan programs, including:
  • Optimization behavior (e.g., should lazy code motion extract constants?)
  • Optimization repetitions
  • Optimization ordering (the current order is my educated guess but is not empirical)
• Making sure that optimizations are applied whenever safe
• 'Optimizing' the optimization code (e.g. some large data structures are recomputed for each optimization)

I want to make it clear that #522 was about solving instances when optimizations produced non-compiling or non-correct code. I solved all issues that arose with our corpus and with all of the optimizations activated. I did not do any performance tuning. In addition, some of the changes I made which decreased the efficacy of the optimizations may have been unnecessarily conservative.

I'd be happy to help out anyone who wanted to work on these, and I'll work on them myself when I have time.

Start by reviewing the optimization behavior, repetitions, ordering, and safety concerns listed in the issue, along with the corpus and optimizations referenced in #522. Profile real Stan programs and compare optimization efficacy and efficiency before deciding what to change. Done means the optimization choices are supported by empirical results and remain safe for the corpus.