cockroachdb / cockroachdb/cockroach
roachtest: perturbation/full/elasticWorkload failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
roachtest.perturbation/full/elasticWorkload [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21430352?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21430352?buildTab=artifacts#/perturbation/full/elasticWorkload) on master @ [37dbe97c5e739eddb6a582c2030fd3d893f539dc](https://github.com/cockroachdb/cockroach/commits/37dbe97c5e739eddb6a582c2030fd3d893f539dc):
Failed with:
```
(assertions.go:387).Fail:
Error Trace: pkg/cmd/roachtest/tests/perturbation/framework.go:1053
pkg/cmd/roachtest/test_runner.go:1853
src/runtime/asm_amd64.s:1771
Error: FAILURE: follower-read : throughput ratio 1.92x exceeds limit 1.2x (baseline: 10985 ops/s, measured: 5708 ops/s)
FAILURE: read : throughput ratio 1.94x exceeds limit 1.2x (baseline: 10992 ops/s, measured: 5668 ops/s)
FAILURE: write : throughput ratio 1.96x exceeds limit 1.2x (baseline: 22120 ops/s, measured: 11305 ops/s)
Test: perturbation/full/elasticWorkload
(require.go:522).Fail: FailNow called
test artifacts and logs in: /artifacts/perturbation/full/elasticWorkload/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21430352-1784524168-201-n13cpu16sm-0001 | 34.138.174.52 | 10.142.0.142 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0002 | 34.26.108.192 | 10.142.0.38 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0003 | 34.74.11.113 | 10.142.0.162 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0004 | 34.148.221.193 | 10.142.0.188 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0005 | 35.237.40.104 | 10.142.0.32 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0006 | 35.190.129.248 | 10.142.0.14 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0007 | 35.231.89.186 | 10.142.0.10 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0008 | 34.24.213.240 | 10.142.0.73 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0009 | 35.227.103.151 | 10.142.0.165 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0010 | 35.196.50.90 | 10.142.0.193 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0011 | 34.139.173.232 | 10.142.0.202 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0012 | 34.74.26.210 | 10.142.0.139 |
| teamcity-21430352-1784524168-201-n13cpu16sm-0013 | 34.73.193.77 | 10.142.0.196 |
Parameters:
- acMode=defaultOption
- arch=amd64
- blockSize=4096
- cloud=gce
- coverageBuild=false
- cpu=16
- diskBandwidthLimit=0
- diskCount=2
- disks=2
- encrypted=false
- fillDuration=10m0s
- fs=ext4
- leaseType=leader
- localSSD=true
- mem=standard
- numNodes=12
- numWorkloadNodes=1
- perturbationDuration=10m0s
- ratioOfMax=0.5
- runtimeAssertionsBuild=false
- seed=0
- splits=10000
- validationDuration=5m0s
- vcpu=16
Help
See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)
See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)
See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21430352/perturbation-full-elasticworkload/1784562930963/1784565516435)
See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21430352-1784524168-201-n13cpu16sm&from_ts=1784562930963&to_ts=1784565516435&live=false&storage=flex_tier)
/cc @cockroachlabs/kv-triage
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*perturbation/full/elasticWorkload.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-65939
Contributor guide
Assessment
This issue has not been assessed yet.