Continuous profiling: leverage `rack.after_reply` ?

One thing I noticed when investigating something with request profiling is that the actual request response time is always a few hundred ms longer than the stackprof or vernier resulting profile.

While both are relatively low overhead when active, I think generating and extracting the profiling data takes a while and impact user experience.

When doing continuous profiling, instead of doing all that from ProfilingMiddleware, we could register a rack.after_reply callback so it's done out of band. Of course of ad hoc profiling, we can't do that because we need to change the response etc.

The advantages are multiple:

• Less impact on the user experience (less latency impact)
• The profile would also include writing the response to the client, so if there is performance to be gained in this part of the server, we'd see it.

cc @dalehamel @bmansoob thoughts ?

Start in ProfilingMiddleware and trace how continuous profiling currently generates and extracts StackProf or Vernier data. Investigate the rack.after_reply callback, keeping ad hoc profiling behavior separate. Done means continuous profiles run out of band, include response writing, and avoid adding their processing latency to the user-facing response.