cockroachdb / cockroachdb/cockroach

basalt-26.3.2-rc-20260908: kv/kvnemesis: TestKVNemesisRangeFlush failed

Open
#175,500 0 comments 0 reactions 0 assignees View on GitHub
branch-basalt-26.3.2-rc-20260908 C-test-failure O-robot release-blocker T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

kv/kvnemesis.TestKVNemesisRangeFlush [failed](https://mesolite.cluster.engflow.com/invocations/default/970da3e6-2720-4b6a-bb2e-546fba63aed9?testReportRun=8&testReportShard=1&testReportAttempt=1#targets-Ly9wa2cva3Yva3ZuZW1lc2lzOmt2bmVtZXNpc190ZXN0) on basalt-26.3.2-rc-20260908 @ [807558e5e81666af257c811e925e4cae570abc28](https://github.com/cockroachdb/cockroach/commits/807558e5e81666af257c811e925e4cae570abc28):

```
=== RUN TestKVNemesisRangeFlush
test_log_scope.go:172: test logs captured to: outputs.zip/logTestKVNemesisRangeFlush848652152
test_log_scope.go:82: use -show-logs to present logs inline
kvnemesis_test.go:797: seed: 2682504684037650345
kvnemesis_test.go:332: n1 using clock[offset=105.326µs,amp=3.394674ms]
kvnemesis_test.go:332: n2 using clock[offset=-153.298µs,amp=3.346702ms]
kvnemesis_test.go:332: n3 using clock[offset=331.972µs,amp=3.168028ms]
test_server_shim.go:146: using test tenant configuration from testserver factory defaults
kvnemesis_test.go:411: kvnemesis logging to /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_8_of_10/test.outputs/kvnemesis4169591154
kvnemesis.go:241: error applying x.AdminMerge(ctx, tk(5500885472691828739)) // merge failed: checking RHS manifest before merge: merge transaction attempting to issue batch on right-hand side range after subsumption: 1 ChkRngSharedManifest: merge failed: checking RHS manifest before merge: merge transaction attempting to issue batch on right-hand side range after subsumption: 1 ChkRngSharedManifest
kvnemesis.go:282: failures(verbose): /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_8_of_10/test.outputs/kvnemesis4169591154/failures
repro steps: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_8_of_10/test.outputs/kvnemesis4169591154/repro.go
rangefeed KVs: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_8_of_10/test.outputs/kvnemesis4169591154/kvs-rangefeed.txt
scan KVs: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_8_of_10/test.outputs/kvnemesis4169591154/kvs-scan.txt
kvnemesis_test.go:927: Metric | Node 1 | Node 2 | Node 3
kvnemesis_test.go:933: ------------------------------------+--------------------------------+--------------------------------+-------------------------------
kvnemesis_test.go:940: follower_reads.success_count | 2 | 0 | 5
kvnemesis_test.go:940: raft.commands.proposed | 2884 | 250 | 61
kvnemesis_test.go:940: raft.commands.reproposed.new-lai | 0 | 2 | 0
kvnemesis_test.go:940: raft.commands.reproposed.unchanged | 128 | 17 | 11
kvnemesis_test.go:940: txn.aborts | 10 | 8 | 26
kvnemesis_test.go:940: txn.commits | 664 | 119 | 112
kvnemesis_test.go:940: txn.durations | μ=20ms p50=4ms p99=386ms | μ=28ms p50=5ms p99=117ms | μ=208ms p50=14ms p99=1.745s
kvnemesis_test.go:940: txn.restarts.writetooold | 1 | 1 | 1
kvnemesis_test.go:940: txn.restarts.serializable | 1 | 0 | 0
kvnemesis_test.go:940: txn.restarts.readwithinuncertainty | 0 | 0 | 0
kvnemesis_test.go:940: txn.server_side.1PC.success | 69 | 4 | 2
kvnemesis_test.go:940: txnrecovery.failures | 0 | 0 | 0
kvnemesis_test.go:940: txnrecovery.successes.aborted | 0 | 0 | 0
kvnemesis_test.go:940: txnrecovery.successes.committed | 0 | 0 | 0
kvnemesis_test.go:940: txnwaitqueue.deadlocks_total | 1 | 3 | 0
kvnemesis_test.go:860:
Error Trace: pkg/kv/kvnemesis/kvnemesis_test.go:860
pkg/kv/kvnemesis/kvnemesis_test.go:782
Error: Should be zero, but was 1
Test: TestKVNemesisRangeFlush
Messages: kvnemesis detected failures
dump_raft_logs.go:32: dumping raft logs to /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_8_of_10/test.outputs/kvnemesis4169591154/raftlogs
panic.go:694: -- test log scope end --
test logs left over in: outputs.zip/logTestKVNemesisRangeFlush848652152
--- FAIL: TestKVNemesisRangeFlush (35.40s)
```

Parameters:
- attempt=1
- run=8
- shard=1
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

Same failure on other branches

- #175404 basalt-26.3.2-rc-20260913: kv/kvnemesis: TestKVNemesisRangeFlush failed [A-kv C-bug C-test-failure O-robot P-2 T-kv branch-basalt-26.3.2-rc-20260913]

/cc @cockroachlabs/kv-triage

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestKVNemesisRangeFlush.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.