release-26.2.6-rc: roachtest: apt_problem failed

roachtest.apt_problem [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyAwsBazel/21473064?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyAwsBazel/21473064?buildTab=artifacts#/cdc/frontier-persistence-benchmark/interval=30s/tables=10000/ranges=1) on release-26.2.6-rc @ [879c22972553ccf2a6e912f0e32d3dd7d5432532](https://github.com/cockroachdb/cockroach/commits/879c22972553ccf2a6e912f0e32d3dd7d5432532):

Failed with:

```
test cdc/frontier-persistence-benchmark/interval=30s/tables=10000/ranges=1 failed: (cdc.go:901).startGrafana: error starting prometheus/grafana: unable to install grafana: error persisted after 3 attempts: TRANSIENT_ERROR(apt_problem): exit status 100
(cdc.go:3613).func32: failed to run "SET CLUSTER SETTING spanconfig.range_coalescing.system.enabled = 'false'": context canceled
(cdc.go:465).Close: error shutting down prometheus/grafana: context canceled
test artifacts and logs in: /artifacts/cdc/frontier-persistence-benchmark/interval=30s/tables=10000/ranges=1/cpu_arch=arm64/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21473064-1787121205-32-n4cpu16-0001 | 18.226.73.157 | 10.12.36.238 |
| teamcity-21473064-1787121205-32-n4cpu16-0002 | 18.118.132.138 | 10.12.39.12 |
| teamcity-21473064-1787121205-32-n4cpu16-0003 | 18.221.74.240 | 10.12.38.41 |
| teamcity-21473064-1787121205-32-n4cpu16-0004 | 3.17.67.106 | 10.12.39.78 |

Parameters:
- arch=arm64
- cloud=aws
- coverageBuild=false
- cpu=16
- diskCount=0
- encrypted=false
- fs=ext4
- localSSD=true
- runtimeAssertionsBuild=false
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

_Grafana is not yet available for aws clusters_

Same failure on other branches

- #173550 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.23-rc]
- #173460 release-26.3.1-rc: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.3.1-rc]
- #172603 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1.8-rc]
- #172566 release-26.3.0-rc: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.3.0-rc]
- #172511 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.22-rc]
- #171789 release-26.3: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.3]
- #171651 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.4.12-rc]
- #171285 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1.5-rc]
- #171154 release-26.2.2-rc: roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.2.2-rc]
- #170470 roachtest: apt_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.19-rc]

/cc @cockroachlabs/test-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*apt_problem.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-66877

Start with the roachtest README and the cdc.go locations reported in the failure, especially startGrafana. Inspect the linked TeamCity logs and artifacts for the apt installation error on the AWS arm64 cluster, then reproduce or compare the listed failures on other branches. Done means the roachtest can install and start Prometheus/Grafana without apt_problem and complete the benchmark.

Written by the indexing model from the issue text.