cockroachdb / cockroachdb/cockroach

roachtest: disk-stalled/wal-failover/among-stores failed

Open
#163,835 3 comments 0 reactions 1 assignee Claimed by @sumeerbhola View on GitHub
A-storage branch-release-25.2 C-test-failure O-roachtest O-robot T-storage
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

roachtest.disk-stalled/wal-failover/among-stores [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21119800?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21119800?buildTab=artifacts#/disk-stalled/wal-failover/among-stores) on release-25.2 @ [41623f2c082b258db3f86bf01f6371ce96245f97](https://github.com/cockroachdb/cockroach/commits/41623f2c082b258db3f86bf01f6371ce96245f97):

```
(disk_stall.go:163).runDiskStalledWALFailover: unexpectedly high p99.99 latency 1.144563174s at 2026-02-18T10:36:00Z
(cluster.go:2451).Run: context canceled
test artifacts and logs in: /artifacts/disk-stalled/wal-failover/among-stores/run_1
```

Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=16
- diskCount=2
- encrypted=false
- fs=ext4
- localSSD=true
- metamorphicBufferedSender=false
- metamorphicLeases=default
- runtimeAssertionsBuild=false
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21119800/disk-stalled-wal-failover-among-stores/1771408331168/1771412448637)

/cc @cockroachdb/storage

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*disk-stalled/wal-failover/among-stores.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-60488

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.