basalt-26.3: roachtest: perturbation/full/partition failed

roachtest.perturbation/full/partition [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21502530?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21502530?buildTab=artifacts#/perturbation/full/partition) on basalt-26.3 @ [30531f831e37dfb219a9b7d87cef860a3b64c094](https://github.com/cockroachdb/cockroach/commits/30531f831e37dfb219a9b7d87cef860a3b64c094):

Failed with:

```
(assertions.go:387).Fail:
Error Trace: pkg/cmd/roachtest/tests/perturbation/framework.go:1051
pkg/cmd/roachtest/test_runner.go:1618
src/runtime/asm_amd64.s:1771
Error: FAILURE: follower-read : throughput ratio 3.68x exceeds limit 1.2x (baseline: 10233 ops/s, measured: 2781 ops/s)
FAILURE: read : throughput ratio 3.74x exceeds limit 1.2x (baseline: 10214 ops/s, measured: 2734 ops/s)
FAILURE: write : throughput ratio 3.70x exceeds limit 1.2x (baseline: 20558 ops/s, measured: 5561 ops/s)
Test: perturbation/full/partition
(require.go:522).Fail: FailNow called
test artifacts and logs in: /artifacts/perturbation/full/partition/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21502530-1788760732-273-n13cpu16sm-0001 | | 10.91.96.147 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0002 | | 10.91.97.74 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0003 | | 10.91.97.72 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0004 | | 10.91.96.188 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0005 | | 10.91.96.214 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0006 | | 10.91.97.69 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0007 | | 10.91.97.67 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0008 | | 10.91.97.71 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0009 | | 10.91.97.75 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0010 | | 10.91.97.65 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0011 | | 10.91.96.128 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0012 | | 10.91.96.146 |
| teamcity-21502530-1788760732-273-n13cpu16sm-0013 | | 10.91.96.135 |

Parameters:
- acMode=defaultOption
- arch=amd64
- blockSize=4096
- cloud=gce
- coverageBuild=false
- cpu=16
- diskBandwidthLimit=0
- diskCount=2
- disks=2
- encrypted=false
- fillDuration=10m0s
- leaseType=expiration
- mem=standard
- numNodes=12
- numWorkloadNodes=1
- perturbationDuration=10m0s
- ratioOfMax=0.5
- runtimeAssertionsBuild=false
- seed=0
- splits=10000
- validationDuration=5m0s
- vcpu=16
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21502530/perturbation-full-partition/1788833448477/1788835698783)

_Logs were not uploaded to Datadog_

Same failure on other branches

- #174740 release-26.3.2-rc: roachtest: perturbation/full/partition failed [C-test-failure O-roachtest O-robot T-kv branch-release-26.3.2-rc release-blocker]
- #173472 release-26.3.1-rc: roachtest: perturbation/full/partition failed [A-testing C-bug C-test-failure O-roachtest O-robot P-3 T-kv branch-release-26.3.1-rc]
- #172639 release-26.3.0-rc: roachtest: perturbation/full/partition failed [C-bug C-test-failure O-roachtest O-robot P-3 T-kv branch-release-26.3.0-rc]
- #171797 release-26.3: roachtest: perturbation/full/partition failed [C-test-failure O-roachtest O-robot P-3 T-kv X-duplicate branch-release-26.3]
- #171399 roachtest: perturbation/full/partition failed [A-testing C-bug C-test-failure O-roachtest O-robot P-2 T-kv branch-master]

/cc @cockroachlabs/kv-triage

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*perturbation/full/partition.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68017

Start with the artifacts and logs at /artifacts/perturbation/full/partition/run_1, then inspect pkg/cmd/roachtest/tests/perturbation/framework.go around line 1051. Compare the Grafana run and the related failures listed in the issue. Done means identifying the cause of the throughput-ratio failure, addressing it, and confirming that roachtest.perturbation/full/partition passes.

Written by the indexing model from the issue text.