Changefeeds potentially slowing down foreground traffic

**Describe the problem**

In our scale testing we observed two issues:
- Foreground p50 latency of the test workload went up by about 5% when starting 6 changefeeds. [Datadog from this time range](https://us5.datadoghq.com/dashboard/pbe-ic2-3qt/drt?fromUser=true&refresh_mode=paused&storage=flex_tier&tpl_var_cluster%5B0%5D=drt-scale-300&tpl_var_monitor_host%5B0%5D=drt-puaaaa%2A&tpl_var_workload_cluster%5B0%5D=workload-scale-300&utc_override=true&from_ts=1765983638661&to_ts=1765987246025&live=false)
- When starting ~60 changefeeds latency went way up (3-4x) to the point of interfering with throughput of the workload. [DD link](https://us5.datadoghq.com/dashboard/pbe-ic2-3qt/drt?fromUser=true&refresh_mode=paused&storage=flex_tier&tpl_var_cluster%5B0%5D=drt-scale-300&tpl_var_monitor_host%5B0%5D=drt-puaaaa%2A&tpl_var_workload_cluster%5B0%5D=workload-scale-300&utc_override=true&from_ts=1766022480000&to_ts=1766024460000&live=false)

**To Reproduce**

It is still somewhat unclear if there's more to the story here than a) having a workload running and b) starting some changefeeds. We haven't yet reproduced.

**Expected behavior**
Changefeeds should not interfere with foreground traffic.

### **Additional data / screenshots**

_We think that buffered sender was off is the likely culprit in this._

**Initial run of 6 feeds that showed this issue**

I started 6 feeds at 2025-12-17 15:32 UTC and we see a small spike in TPCC workload p50 latencies more or less immediately. [Datadog from this time range](https://us5.datadoghq.com/dashboard/pbe-ic2-3qt/drt?fromUser=true&refresh_mode=paused&storage=flex_tier&tpl_var_cluster%5B0%5D=drt-scale-300&tpl_var_monitor_host%5B0%5D=drt-puaaaa%2A&tpl_var_workload_cluster%5B0%5D=workload-scale-300&utc_override=true&from_ts=1765983638661&to_ts=1765987246025&live=false)

We did see that buffered sender was off (with SHOW CLUSTER SETTING) which it is by default in 25.4 (but not 25.3 or 26.1). We see that there is an accompanying spike in goroutines. I am not seeing a spike in the scheduler latency at this time, but I think that is likely because of issues configuring the dashboard.

**Trying to confirm issue goes away when turning off feeds**

At 2025-12-18 01:19 UTC I turned off webhook and cloudstorage feeds, but saw no difference in P50, p90, p99 workload latencies as of 2025-12-18 01:24 UTC. (Even though above changes were pretty immediate). Then I canceled the kafka changefeed at 2025-12-18 01:25 UTC and also did not see a perceptible difference. Then we had a *spike* in P50, p90, p99 workload latencies at 2025-12-18 01:30 UTC even though we’re running fewer changefeeds. At 2025-12-18 01:38 UTC cancel remaining changefeeds.

Stan suggested: 1) there was a RangeStats storm, 2) “That could just be the side-effect of the workload transitioning from closed to open, since presumably changefeed was backpressuring it.”
If we do think that turning on changefeeds affects latency, we did not see it in this instance.
[Datadog](https://us5.datadoghq.com/dashboard/pbe-ic2-3qt/drt?fromUser=true&refresh_mode=paused&storage=flex_tier&tpl_var_cluster%5B0%5D=drt-scale-300&tpl_var_monitor_host%5B0%5D=drt-puaaaa%2A&tpl_var_workload_cluster%5B0%5D=workload-scale-300&utc_override=true&from_ts=1766020090812&to_ts=1766022496341&live=false) [Slack](https://cockroachlabs.slack.com/archives/C0A0KECF60Y/p1766021787239609?thread_ts=1765999961.232009&cid=C0A0KECF60Y)

**58 feeds**

See this [DD link](https://us5.datadoghq.com/dashboard/pbe-ic2-3qt/drt?fromUser=true&refresh_mode=paused&storage=flex_tier&tpl_var_cluster%5B0%5D=drt-scale-300&tpl_var_monitor_host%5B0%5D=drt-puaaaa%2A&tpl_var_workload_cluster%5B0%5D=workload-scale-300&utc_override=true&from_ts=1766022480000&to_ts=1766024460000&live=false) for more info but

Attempted to create 300 feeds starting at 2025-12-18 01:49 UTC. At 2025-12-18 01:53 UTC: only got to 59 feeds and I stopped making new feeds since we were seeing big issues with foreground latency. Explosion in commit latency, max behind nanos, goroutines etc. I canceled remaining feeds at 2025-12-18 02:09 UTC, (one range unavailable, many underreplicated). At 2025-12-18 02:19 UTC are mostly back to normal (but I do think that latency was ever so slightly higher than where it was before even though no changefeeds were running).

**Other things to note**

- Scale test workload had closed and open modes and switching between them may be a result of changefeeds starting. There is a workload complication: https://cockroachlabs.slack.com/archives/C0A0KECF60Y/p1766021787239609?thread_ts=1765999961.232009&cid=C0A0KECF60Y

**Environment:**
Dec 2025 scale test.

**Additional context**
What was the impact?

Add any other context about the problem here.

Jira issue: CRDB-58144