cockroachdb / cockroachdb/cockroach

roachtest: perturbation/full/backup failed

Open
#170,480 9 comments 0 reactions 0 assignees View on GitHub
A-testing branch-release-25.4 C-bug C-test-failure O-perturbation O-roachtest O-robot P-3 T-kv X-duplicate
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

roachtest.perturbation/full/backup [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21350934?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21350934?buildTab=artifacts#/perturbation/full/backup) on release-25.4 @ [12d38425f6027fbb20aeddd56424e7805e89bb5a](https://github.com/cockroachdb/cockroach/commits/12d38425f6027fbb20aeddd56424e7805e89bb5a):

```
(assertions.go:369).Fail:
Error Trace: pkg/cmd/roachtest/tests/perturbation/framework.go:717
pkg/cmd/roachtest/test_runner.go:1457
src/runtime/asm_amd64.s:1700
Error: Should be true
Test: perturbation/full/backup
Messages: FAILURE: follower-read : Increase 6.6590 > 5.0000 BASE: 5.255803ms SCORE: 34.998203ms

FAILURE: read : Increase 6.5794 > 5.0000 BASE: 5.423392ms SCORE: 35.682551ms
(require.go:2014).True: FailNow called
test artifacts and logs in: /artifacts/perturbation/full/backup/run_1
```

Parameters:
- acMode=defaultOption
- arch=amd64
- blockSize=4096
- cloud=gce
- coverageBuild=false
- cpu=16
- diskBandwidthLimit=0
- disks=2
- encrypted=false
- fillDuration=10m0s
- fs=ext4
- leaseType=leader
- localSSD=true
- mem=standard
- numNodes=12
- numWorkloadNodes=1
- perturbationDuration=10m0s
- ratioOfMax=0.5
- runtimeAssertionsBuild=false
- seed=0
- splits=10000
- ssd=2
- validationDuration=5m0s
- vcpu=16
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21350934/perturbation-full-backup/1779109009687/1779110695721)

Same failure on other branches

- #168321 roachtest: perturbation/full/backup failed [A-testing C-bug C-test-failure O-roachtest O-robot P-3 T-kv branch-release-25.2]
- #167525 roachtest: perturbation/full/backup failed [A-testing C-bug C-test-failure O-perturbation O-roachtest O-robot P-3 T-kv branch-release-26.1 no-test-failure-activity]

/cc @cockroachdb/kv-triage

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*perturbation/full/backup.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-64027

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.