Investigate / reduce overhead preventing higher CPU utilization

Max CPU consumed by a cluster generally maxes out at ~75%, even with a plain KV workload that shouldn't hit I/O bottlenecks. This seems do be due to a bunch of overhead in our runtime stack. E.g. Refer to the screenshot below taken on a KV workload.


This is a concern to customers because we charge for 100% of the cluster, but from the customer's perspective, they can't use 100% of the cluster. Additionally, it puts us at a price/performance disadvantage vs competitors that exhibit less overhead. We should look at these sources of overhead and determine if it makes sense to / if there is opportunity to optimize anything.

More context is here: https://cockroachlabs.slack.com/archives/C04HQCNHGEP/p1699962493375829

Jira issue: CRDB-33572