server: dump execution traces on demand via flight recording

**Is your feature request related to a problem? Please describe.**

When CockroachDB experiences runtime anomalies - CPU spikes, scheduling latency breaches, overload conditions, or other transient performance issues - execution traces are one of the most powerful diagnostic tools available. However, by the time we *recognize* that something interesting has happened, the window of interest is usually already in the past. Today, execution traces must be collected proactively (e.g. via a cluster setting added in https://github.com/cockroachdb/cockroach/pull/149373, or manually via `pprof-loop`), and there is no mechanism to automatically retain traces that overlap with an interesting event.

This is the execution trace analog of on-demand goroutine dumps (https://github.com/cockroachdb/cockroach/issues/159342, https://github.com/cockroachdb/cockroach/pull/159352): when something "interesting" happens, we want to be able to dump an execution trace that covers the event. Unlike goroutine dumps, however, execution traces must have been *actively recording* at the time of the event, since they capture scheduling decisions, GC activity, and goroutine state transitions over a time window. This means the feature fundamentally depends on some form of **flight recording** - maintaining recent execution trace data in memory and persisting it only when a trigger fires.

Multiple recent support escalations have highlighted the need for this:

- In cockroachlabs/support#3483, investigation of periodic CPU microspikes causing elastic work throttling relied heavily on execution traces. The root cause (coalesced heartbeat-induced scheduling latency on low-vCPU machines) was diagnosed through manual execution trace analysis, but the turnaround time was significantly extended by the difficulty of obtaining overlapping traces. The elastic CPU controller's 1ms scheduling latency threshold is a natural trigger: if we could automatically dump an execution trace whenever this threshold is breached, we'd have the artifacts needed to diagnose the root cause without round-trips to the customer.
- In cockroachlabs/support#3548, a multi-hour cluster-wide outage featured high p99 scheduling latency and downstream effects (StoreLiveness heartbeat failures, lost leases), but no execution traces were available from the outage window. Flight-recorded execution traces dumped on a scheduling latency trigger would have provided immediate visibility into what was consuming runtime resources.

**Describe the solution you'd like**

A mechanism to **automatically persist execution traces when something interesting happens**, covering a time window that includes the triggering event.

At a high level, this requires two things:

1. **Execution trace data must be available for the recent past.** The simplest way to achieve this is always-on [flight recording](https://github.com/golang/go/issues/63185), where the Go runtime continuously records execution trace data into a bounded in-memory buffer (e.g. the last 10-30 seconds). Alternatively, the existing periodic execution trace collection (https://github.com/cockroachdb/cockroach/pull/149373) could evolve to keep traces in memory rather than immediately persisting to disk, achieving a similar effect. Either way, the goal is that when a trigger fires, there is trace data covering the event.

2. **Triggers that cause the trace data to be persisted.** Various subsystems should be able to request a dump. Initial candidates include:
- The **elastic CPU controller** (admission control), when the scheduling latency threshold triggers.
- The **store liveness / node liveness** layer, when heartbeat failures or suspect node conditions are detected.
- Any **overload detection** mechanism.

The persisted traces should be stored alongside other observability artifacts (heap profiles, CPU profiles, goroutine dumps), subject to configurable retention policies, and retrievable via existing APIs (including `debug zip`).

**Describe alternatives you've considered**

- **Manual collection only**: The status quo. Requires customers to run `pprof-loop` or use cluster settings (https://github.com/cockroachdb/cockroach/pull/149373), and doesn't solve the "event is in the past" problem.
- **Continuous trace collection to disk**: Already possible via https://github.com/cockroachdb/cockroach/pull/149373. Produces large volumes of data, most of which is uninteresting, still requires manual correlation with events, and critically, is unlikely to already be enabled when something interesting happens - requiring a time-consuming cycle of asking the customer to turn it on, hoping the problem reoccurs, and then engaging with the collected data.

**Relationship to other work**

- https://github.com/cockroachdb/cockroach/issues/161500 tracks integrating execution traces with **statement diagnostics bundles** - a complementary effort focused on correlating execution traces with specific slow statement executions. A key distinction is that diagnostics bundles don't strictly require flight recording: the diagnostics request is made *before* the statement of interest executes, so execution tracing can be started proactively and run forward. In contrast, this issue is about reacting to events that have *already happened*, which fundamentally requires looking into the past. That said, flight recording infrastructure built for this issue would also benefit #161500.
- https://github.com/cockroachdb/cockroach/pull/146873 is an earlier prototype (`CoalescedFlightRecorder`) that demonstrates one possible approach to the infrastructure described here.
- https://github.com/cockroachdb/cockroach/pull/146650 is a prototype for using the flight recorder specifically during CPU overload.
- https://github.com/cockroachdb/cockroach/issues/159342 / https://github.com/cockroachdb/cockroach/pull/159352 established the **on-demand goroutine dump** pattern (trigger-based, rate-limited, plumbed to KV via `StoreConfig`). The execution trace dump mechanism should follow a similar architectural pattern.
- https://github.com/cockroachdb/cockroach/issues/97215 was the original issue for integrating execution tracing with CRDB, closed before the desired functionality was fully realized.
- https://github.com/cockroachdb/cockroach/issues/129942 (kvstreamer scheduling latency impact) is an example of the class of issue where automatic trace dumps would help.

**Additional context**

Go execution trace overhead has been [significantly reduced](https://go.dev/blog/execution-traces-2024) since Go 1.21, making continuous flight recording increasingly feasible. Overhead characterization for CockroachDB workloads would be useful early work toward this.

Jira issue: CRDB-60151