cockroachdb / cockroachdb/cockroach

basalt-26.3.2-rc-20260913: kv/kvnemesis: TestKVNemesisRangeFlush failed

Open
#175,404 3 comments 0 reactions 1 assignee Claimed by @iskettaneh View on GitHub
A-kv branch-basalt-26.3.2-rc-20260913 C-bug C-test-failure O-robot P-2 T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

kv/kvnemesis.TestKVNemesisRangeFlush [failed](https://mesolite.cluster.engflow.com/invocations/default/7559051b-e99a-48c2-8931-589ea459e36a?testReportRun=3&testReportShard=1&testReportAttempt=1#targets-Ly9wa2cva3Yva3ZuZW1lc2lzOmt2bmVtZXNpc190ZXN0) on basalt-26.3.2-rc-20260913 @ [c6a822fb5ec0b622c713fccacb82159d1b60bbcd](https://github.com/cockroachdb/cockroach/commits/c6a822fb5ec0b622c713fccacb82159d1b60bbcd):

```
=== RUN TestKVNemesisRangeFlush
test_log_scope.go:172: test logs captured to: outputs.zip/logTestKVNemesisRangeFlush2821126838
test_log_scope.go:82: use -show-logs to present logs inline
kvnemesis_test.go:797: seed: 7518940720771069854
kvnemesis_test.go:332: n1 using clock[offset=-313.761µs,amp=3.186239ms]
kvnemesis_test.go:332: n2 using clock[offset=-120.52µs,amp=3.37948ms]
kvnemesis_test.go:332: n3 using clock[offset=252.752µs,amp=3.247248ms]
test_server_shim.go:146: using test tenant configuration from testserver factory defaults
kvnemesis_test.go:411: kvnemesis logging to /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_3_of_3/test.outputs/kvnemesis3557478295
kvnemesis.go:241: error applying x.AdminMerge(ctx, tk(4709558071917714632)) // merge failed: checking RHS manifest before merge: merge transaction attempting to issue batch on right-hand side range after subsumption: 1 ChkRngSharedManifest: merge failed: checking RHS manifest before merge: merge transaction attempting to issue batch on right-hand side range after subsumption: 1 ChkRngSharedManifest
kvnemesis.go:282: failures(verbose): /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_3_of_3/test.outputs/kvnemesis3557478295/failures
repro steps: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_3_of_3/test.outputs/kvnemesis3557478295/repro.go
rangefeed KVs: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_3_of_3/test.outputs/kvnemesis3557478295/kvs-rangefeed.txt
scan KVs: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_3_of_3/test.outputs/kvnemesis3557478295/kvs-scan.txt
kvnemesis_test.go:927: Metric | Node 1 | Node 2 | Node 3
kvnemesis_test.go:933: ------------------------------------+--------------------------------+--------------------------------+-------------------------------
kvnemesis_test.go:940: follower_reads.success_count | 0 | 1 | 1
kvnemesis_test.go:940: raft.commands.proposed | 3482 | 20 | 52
kvnemesis_test.go:940: raft.commands.reproposed.new-lai | 17 | 0 | 0
kvnemesis_test.go:940: raft.commands.reproposed.unchanged | 176 | 13 | 24
kvnemesis_test.go:940: txn.aborts | 27 | 35 | 16
kvnemesis_test.go:940: txn.commits | 656 | 133 | 201
kvnemesis_test.go:940: txn.durations | μ=29ms p50=11ms p99=434ms | μ=109ms p50=9ms p99=761ms | μ=100ms p50=10ms p99=962ms
kvnemesis_test.go:940: txn.restarts.writetooold | 2 | 0 | 4
kvnemesis_test.go:940: txn.restarts.serializable | 1 | 0 | 0
kvnemesis_test.go:940: txn.restarts.readwithinuncertainty | 0 | 0 | 0
kvnemesis_test.go:940: txn.server_side.1PC.success | 86 | 0 | 0
kvnemesis_test.go:940: txnrecovery.failures | 0 | 0 | 0
kvnemesis_test.go:940: txnrecovery.successes.aborted | 1 | 0 | 0
kvnemesis_test.go:940: txnrecovery.successes.committed | 0 | 0 | 0
kvnemesis_test.go:940: txnwaitqueue.deadlocks_total | 13 | 0 | 0
kvnemesis_test.go:860:
Error Trace: pkg/kv/kvnemesis/kvnemesis_test.go:860
pkg/kv/kvnemesis/kvnemesis_test.go:782
Error: Should be zero, but was 1
Test: TestKVNemesisRangeFlush
Messages: kvnemesis detected failures
dump_raft_logs.go:32: dumping raft logs to /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/kv/kvnemesis/kvnemesis_test/run_3_of_3/test.outputs/kvnemesis3557478295/raftlogs
panic.go:694: -- test log scope end --
test logs left over in: outputs.zip/logTestKVNemesisRangeFlush2821126838
--- FAIL: TestKVNemesisRangeFlush (41.75s)
```

Parameters:
- attempt=1
- deadlock=true
- run=3
- shard=1
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

/cc @cockroachlabs/kv-triage

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestKVNemesisRangeFlush.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.