roachtest: cdc/frontier-persistence-benchmark/interval=30s/tables=100/ranges=100 failed

roachtest.cdc/frontier-persistence-benchmark/interval=30s/tables=100/ranges=100 [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyAzureBazel/21511271?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyAzureBazel/21511271?buildTab=artifacts#/cdc/frontier-persistence-benchmark/interval=30s/tables=100/ranges=100) on master @ [b503dd4428002dbead4907472e2121ed6bd70c8c](https://github.com/cockroachdb/cockroach/commits/b503dd4428002dbead4907472e2121ed6bd70c8c):

Failed with:

```
(monitor.go:329).Wait: monitor failure: changefeed[default]: max latency was more than allowed: 2m0.027666416s vs 2m0s
test artifacts and logs in: /artifacts/cdc/frontier-persistence-benchmark/interval=30s/tables=100/ranges=100/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21511271-1789105270-30-n4cpu16-0001 | 52.188.225.131 | 10.1.0.132 |
| teamcity-21511271-1789105270-30-n4cpu16-0002 | 40.90.252.77 | 10.1.0.140 |
| teamcity-21511271-1789105270-30-n4cpu16-0003 | 20.55.89.112 | 10.1.0.14 |
| teamcity-21511271-1789105270-30-n4cpu16-0004 | 172.178.85.209 | 10.1.0.103 |

Parameters:
- arch=amd64
- changefeed.aggregator.periodic_flushing.enabled=true
- changefeed.progress.per_table_tracking.enabled=false
- cloud=azure
- coverageBuild=false
- cpu=16
- diskCount=0
- encrypted=false
- fs=ext4
- kv.rangefeed.scheduler.enabled=true
- localSSD=true
- runtimeAssertionsBuild=false
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21511271/cdc-frontier-persistence-benchmark-interval-30s-tables-100-rang/1789106601139/1789108852913)

See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21511271-1789105270-30-n4cpu16&from_ts=1789106601139&to_ts=1789108852913&live=false&storage=flex_tier)

/cc @cockroachlabs/cdc

[Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68177

Start with the roachtest README, the failure at monitor.go:329, and the linked TeamCity logs and artifacts for the frontier-persistence-benchmark run. Compare the Grafana and Datadog data around the reported interval and determine why maximum latency exceeded the two-minute limit. Done means the underlying failure is addressed and this benchmark completes within its allowed latency.

Written by the indexing model from the issue text.