cockroachdb / cockroachdb/cockroach
kv/kvserver: TestCheckConsistencyInconsistent failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
kv/kvserver.TestCheckConsistencyInconsistent [failed](https://mesolite.cluster.engflow.com/invocations/default/148dd7f6-f2f3-4f2f-bae8-4c35fdbf475f?testReportRun=1&testReportShard=12&testReportAttempt=1#targets-Ly9wa2cva3Yva3ZzZXJ2ZXI6a3ZzZXJ2ZXJfdGVzdA==) on release-25.4.17-rc @ [3981ecae25dbf445f122d375bf813b8859158a14](https://github.com/cockroachdb/cockroach/commits/3981ecae25dbf445f122d375bf813b8859158a14):
Fatal error:
```
panic: during (fs.Env).Close there remain open files; all files must be closed before closing the Env
1 open files:
github.com/cockroachdb/pebble/vfs/vfstest.(*openFilesFS).Create
external/com_github_cockroachdb_pebble/vfs/vfstest/open_files.go:51
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create.func1
external/com_github_cockroachdb_pebble/vfs/disk_health.go:748
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).timeFilesystemOp
external/com_github_cockroachdb_pebble/vfs/disk_health.go:655
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_health.go:747
github.com/cockroachdb/pebble/vfs.(*enospcFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_full.go:193
github.com/cockroachdb/cockroach/pkg/storage/fs.(*Env).Create
pkg/storage/fs/fs.go:371
github.com/cockroachdb/pebble/vfs.(*syncingFS).Create
external/com_github_cockroachdb_pebble/vfs/syncing_file.go:205
github.com/cockroachdb/pebble/wal.Copy
external/com_github_cockroachdb_pebble/wal/reader.go:399
github.com/cockroachdb/pebble.(*DB).Checkpoint
external/com_github_cockroachdb_pebble/checkpoint.go:418
github.com/cockroachdb/cockroach/pkg/storage.(*Pebble).CreateCheckpoint
pkg/storage/pebble.go:2388
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).checkpoint
pkg/kv/kvserver/store.go:3716
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).computeChecksumPostApply
pkg/kv/kvserver/replica_consistency.go:797
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleComputeChecksumResult
pkg/kv/kvserver/replica_application_result.go:651
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).handleNonTrivialReplicatedEvalResult
pkg/kv/kvserver/replica_application_state_machine.go:353
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).ApplySideEffects
pkg/kv/kvserver/replica_application_state_machine.go:198
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.mapCheckedCmdIter
pkg/kv/kvserver/apply/cmd.go:209
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).applyOneBatch
pkg/kv/kvserver/apply/task.go:290
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).ApplyCommittedEntries
pkg/kv/kvserver/apply/task.go:246
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked
pkg/kv/kvserver/replica_raft.go:1239
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady
pkg/kv/kvserver/replica_raft.go:874
[recovered]
panic: during (fs.Env).Close there remain open files; all files must be closed before closing the Env
1 open files:
github.com/cockroachdb/pebble/vfs/vfstest.(*openFilesFS).Create
external/com_github_cockroachdb_pebble/vfs/vfstest/open_files.go:51
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create.func1
external/com_github_cockroachdb_pebble/vfs/disk_health.go:748
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).timeFilesystemOp
external/com_github_cockroachdb_pebble/vfs/disk_health.go:655
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_health.go:747
github.com/cockroachdb/pebble/vfs.(*enospcFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_full.go:193
github.com/cockroachdb/cockroach/pkg/storage/fs.(*Env).Create
pkg/storage/fs/fs.go:371
github.com/cockroachdb/pebble/vfs.(*syncingFS).Create
external/com_github_cockroachdb_pebble/vfs/syncing_file.go:205
github.com/cockroachdb/pebble/wal.Copy
external/com_github_cockroachdb_pebble/wal/reader.go:399
github.com/cockroachdb/pebble.(*DB).Checkpoint
external/com_github_cockroachdb_pebble/checkpoint.go:418
github.com/cockroachdb/cockroach/pkg/storage.(*Pebble).CreateCheckpoint
pkg/storage/pebble.go:2388
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).checkpoint
pkg/kv/kvserver/store.go:3716
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).computeChecksumPostApply
pkg/kv/kvserver/replica_consistency.go:797
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleComputeChecksumResult
pkg/kv/kvserver/replica_application_result.go:651
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).handleNonTrivialReplicatedEvalResult
pkg/kv/kvserver/replica_application_state_machine.go:353
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).ApplySideEffects
pkg/kv/kvserver/replica_application_state_machine.go:198
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.mapCheckedCmdIter
pkg/kv/kvserver/apply/cmd.go:209
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).applyOneBatch
pkg/kv/kvserver/apply/task.go:290
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).ApplyCommittedEntries
pkg/kv/kvserver/apply/task.go:246
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked
pkg/kv/kvserver/replica_raft.go:1239
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady
pkg/kv/kvserver/replica_raft.go:874
[recovered]
panic: during (fs.Env).Close there remain open files; all files must be closed before closing the Env
1 open files:
github.com/cockroachdb/pebble/vfs/vfstest.(*openFilesFS).Create
external/com_github_cockroachdb_pebble/vfs/vfstest/open_files.go:51
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create.func1
external/com_github_cockroachdb_pebble/vfs/disk_health.go:748
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).timeFilesystemOp
external/com_github_cockroachdb_pebble/vfs/disk_health.go:655
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_health.go:747
github.com/cockroachdb/pebble/vfs.(*enospcFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_full.go:193
github.com/cockroachdb/cockroach/pkg/storage/fs.(*Env).Create
pkg/storage/fs/fs.go:371
github.com/cockroachdb/pebble/vfs.(*syncingFS).Create
external/com_github_cockroachdb_pebble/vfs/syncing_file.go:205
github.com/cockroachdb/pebble/wal.Copy
external/com_github_cockroachdb_pebble/wal/reader.go:399
github.com/cockroachdb/pebble.(*DB).Checkpoint
external/com_github_cockroachdb_pebble/checkpoint.go:418
github.com/cockroachdb/cockroach/pkg/storage.(*Pebble).CreateCheckpoint
pkg/storage/pebble.go:2388
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).checkpoint
pkg/kv/kvserver/store.go:3716
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).computeChecksumPostApply
pkg/kv/kvserver/replica_consistency.go:797
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleComputeChecksumResult
pkg/kv/kvserver/replica_application_result.go:651
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).handleNonTrivialReplicatedEvalResult
pkg/kv/kvserver/replica_application_state_machine.go:353
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).ApplySideEffects
pkg/kv/kvserver/replica_application_state_machine.go:198
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.mapCheckedCmdIter
pkg/kv/kvserver/apply/cmd.go:209
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).applyOneBatch
pkg/kv/kvserver/apply/task.go:290
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).ApplyCommittedEntries
pkg/kv/kvserver/apply/task.go:246
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked
pkg/kv/kvserver/replica_raft.go:1239
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady
pkg/kv/kvserver/replica_raft.go:874
[recovered]
panic: during (fs.Env).Close there remain open files; all files must be closed before closing the Env
1 open files:
github.com/cockroachdb/pebble/vfs/vfstest.(*openFilesFS).Create
external/com_github_cockroachdb_pebble/vfs/vfstest/open_files.go:51
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create.func1
external/com_github_cockroachdb_pebble/vfs/disk_health.go:748
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).timeFilesystemOp
external/com_github_cockroachdb_pebble/vfs/disk_health.go:655
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_health.go:747
github.com/cockroachdb/pebble/vfs.(*enospcFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_full.go:193
github.com/cockroachdb/cockroach/pkg/storage/fs.(*Env).Create
pkg/storage/fs/fs.go:371
github.com/cockroachdb/pebble/vfs.(*syncingFS).Create
external/com_github_cockroachdb_pebble/vfs/syncing_file.go:205
github.com/cockroachdb/pebble/wal.Copy
external/com_github_cockroachdb_pebble/wal/reader.go:399
github.com/cockroachdb/pebble.(*DB).Checkpoint
external/com_github_cockroachdb_pebble/checkpoint.go:418
github.com/cockroachdb/cockroach/pkg/storage.(*Pebble).CreateCheckpoint
pkg/storage/pebble.go:2388
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).checkpoint
pkg/kv/kvserver/store.go:3716
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).computeChecksumPostApply
pkg/kv/kvserver/replica_consistency.go:797
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleComputeChecksumResult
pkg/kv/kvserver/replica_application_result.go:651
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).handleNonTrivialReplicatedEvalResult
pkg/kv/kvserver/replica_application_state_machine.go:353
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).ApplySideEffects
pkg/kv/kvserver/replica_application_state_machine.go:198
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.mapCheckedCmdIter
pkg/kv/kvserver/apply/cmd.go:209
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).applyOneBatch
pkg/kv/kvserver/apply/task.go:290
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).ApplyCommittedEntries
pkg/kv/kvserver/apply/task.go:246
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked
pkg/kv/kvserver/replica_raft.go:1239
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady
pkg/kv/kvserver/replica_raft.go:874
[recovered]
panic: during (fs.Env).Close there remain open files; all files must be closed before closing the Env
1 open files:
github.com/cockroachdb/pebble/vfs/vfstest.(*openFilesFS).Create
external/com_github_cockroachdb_pebble/vfs/vfstest/open_files.go:51
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create.func1
external/com_github_cockroachdb_pebble/vfs/disk_health.go:748
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).timeFilesystemOp
external/com_github_cockroachdb_pebble/vfs/disk_health.go:655
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_health.go:747
github.com/cockroachdb/pebble/vfs.(*enospcFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_full.go:193
github.com/cockroachdb/cockroach/pkg/storage/fs.(*Env).Create
pkg/storage/fs/fs.go:371
github.com/cockroachdb/pebble/vfs.(*syncingFS).Create
external/com_github_cockroachdb_pebble/vfs/syncing_file.go:205
github.com/cockroachdb/pebble/wal.Copy
external/com_github_cockroachdb_pebble/wal/reader.go:399
github.com/cockroachdb/pebble.(*DB).Checkpoint
external/com_github_cockroachdb_pebble/checkpoint.go:418
github.com/cockroachdb/cockroach/pkg/storage.(*Pebble).CreateCheckpoint
pkg/storage/pebble.go:2388
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).checkpoint
pkg/kv/kvserver/store.go:3716
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).computeChecksumPostApply
pkg/kv/kvserver/replica_consistency.go:797
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleComputeChecksumResult
pkg/kv/kvserver/replica_application_result.go:651
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).handleNonTrivialReplicatedEvalResult
pkg/kv/kvserver/replica_application_state_machine.go:353
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).ApplySideEffects
pkg/kv/kvserver/replica_application_state_machine.go:198
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.mapCheckedCmdIter
pkg/kv/kvserver/apply/cmd.go:209
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).applyOneBatch
pkg/kv/kvserver/apply/task.go:290
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).ApplyCommittedEntries
pkg/kv/kvserver/apply/task.go:246
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked
pkg/kv/kvserver/replica_raft.go:1239
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady
pkg/kv/kvserver/replica_raft.go:874
```
Stack:
```
goroutine 57110 [running]:
testing.tRunner.func1.2({0xb9d2e40, 0xc008ae11b0})
GOROOT/src/testing/testing.go:1632 +0x3fc
testing.tRunner.func1()
GOROOT/src/testing/testing.go:1635 +0x6b6
panic({0xb9d2e40?, 0xc008ae11b0?})
GOROOT/src/runtime/panic.go:791 +0x132
github.com/cockroachdb/cockroach/pkg/util/leaktest.AfterTest.func2()
pkg/util/leaktest/leaktest.go:136 +0x51a
panic({0xb9d2e40?, 0xc008ae11b0?})
GOROOT/src/runtime/panic.go:791 +0x132
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).recover(0xc002314180?, {0xdb11d28, 0x13fb3b00})
pkg/util/stop/stopper.go:226 +0x74
panic({0xb9d2e40?, 0xc008ae11b0?})
GOROOT/src/runtime/panic.go:791 +0x132
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).recover(0xc00ee9a2b8?, {0xdb12230, 0x13fb3b00})
pkg/util/stop/stopper.go:226 +0x74
panic({0xb9d2e40?, 0xc008ae11b0?})
GOROOT/src/runtime/panic.go:791 +0x132
github.com/cockroachdb/cockroach/pkg/storage/fs.InitEnv.func1()
pkg/storage/fs/fs.go:154 +0x14a
github.com/cockroachdb/cockroach/pkg/storage/fs.(*Env).Close(0xc013f739a0)
pkg/storage/fs/fs.go:314 +0x4d5
github.com/cockroachdb/cockroach/pkg/storage.(*Pebble).Close(0xc009ee7c08)
pkg/storage/pebble.go:1489 +0x20f
github.com/cockroachdb/cockroach/pkg/server.(*Engines).Close(0xc00ef86570)
pkg/server/config.go:689 +0x7c
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).Stop(0xc013f73900, {0xdb12230, 0x13fb3b00})
pkg/util/stop/stopper.go:607 +0x262
github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).stopServerLocked(0xc007c55108, 0x2)
pkg/testutils/testcluster/testcluster.go:244 +0xe5
github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).stopServers(0xc007c55108, {0xdb12230, 0x13fb3b00})
pkg/testutils/testcluster/testcluster.go:181 +0x42e
github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).Start.func3()
pkg/testutils/testcluster/testcluster.go:542 +0x3d
github.com/cockroachdb/cockroach/pkg/util/stop.CloserFn.Close(0xc014186890)
pkg/util/stop/stopper.go:103 +0x27
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).Stop(0xc01601e3c0, {0xdb11d28, 0x13fb3b00})
pkg/util/stop/stopper.go:607 +0x262
runtime.Goexit()
GOROOT/src/runtime/panic.go:635 +0x5e
testing.(*common).FailNow(0xc002d681a0)
GOROOT/src/testing/testing.go:1006 +0x85
testing.(*common).Fatalf(0xc002d681a0, {0xc2b6699, 0x2a}, {0xc009308ec0, 0x2, 0x2})
GOROOT/src/testing/testing.go:1090 +0xaf
github.com/cockroachdb/cockroach/pkg/testutils.SucceedsWithin({0xdae7970, 0xc002d681a0}, 0xc011819870, 0x34630b8a00)
pkg/testutils/soon.go:60 +0x251
github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoon({0xdae7970, 0xc002d681a0}, 0xc011819870)
pkg/testutils/soon.go:38 +0x55
github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestCheckConsistencyInconsistent(0xc002d681a0)
pkg/kv/kvserver/consistency_queue_test.go:384 +0x19da
testing.tRunner(0xc002d681a0, 0xc65a5d8)
GOROOT/src/testing/testing.go:1690 +0x227
created by testing.(*T).Run in goroutine 1
GOROOT/src/testing/testing.go:1743 +0x826
```
Log preceding fatal error
```
stopper.go:226: -- test log scope end --
ERROR: a panic has occurred!
Details cannot be printed yet because we are still unwinding.
Hopefully the test harness prints the panic below, otherwise check the test logs.
test logs left over in: outputs.zip/logTestCheckConsistencyInconsistent4070634102
stopper.go:226: panic: during (fs.Env).Close there remain open files; all files must be closed before closing the Env
1 open files:
github.com/cockroachdb/pebble/vfs/vfstest.(*openFilesFS).Create
external/com_github_cockroachdb_pebble/vfs/vfstest/open_files.go:51
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create.func1
external/com_github_cockroachdb_pebble/vfs/disk_health.go:748
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).timeFilesystemOp
external/com_github_cockroachdb_pebble/vfs/disk_health.go:655
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_health.go:747
github.com/cockroachdb/pebble/vfs.(*enospcFS).Create
external/com_github_cockroachdb_pebble/vfs/disk_full.go:193
github.com/cockroachdb/cockroach/pkg/storage/fs.(*Env).Create
pkg/storage/fs/fs.go:371
github.com/cockroachdb/pebble/vfs.(*syncingFS).Create
external/com_github_cockroachdb_pebble/vfs/syncing_file.go:205
github.com/cockroachdb/pebble/wal.Copy
external/com_github_cockroachdb_pebble/wal/reader.go:399
github.com/cockroachdb/pebble.(*DB).Checkpoint
external/com_github_cockroachdb_pebble/checkpoint.go:418
github.com/cockroachdb/cockroach/pkg/storage.(*Pebble).CreateCheckpoint
pkg/storage/pebble.go:2388
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).checkpoint
pkg/kv/kvserver/store.go:3716
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).computeChecksumPostApply
pkg/kv/kvserver/replica_consistency.go:797
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleComputeChecksumResult
pkg/kv/kvserver/replica_application_result.go:651
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).handleNonTrivialReplicatedEvalResult
pkg/kv/kvserver/replica_application_state_machine.go:353
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).ApplySideEffects
pkg/kv/kvserver/replica_application_state_machine.go:198
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.mapCheckedCmdIter
pkg/kv/kvserver/apply/cmd.go:209
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).applyOneBatch
pkg/kv/kvserver/apply/task.go:290
github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).ApplyCommittedEntries
pkg/kv/kvserver/apply/task.go:246
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked
pkg/kv/kvserver/replica_raft.go:1239
github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady
pkg/kv/kvserver/replica_raft.go:874
--- FAIL: TestCheckConsistencyInconsistent (261.40s)
```
Parameters:
- attempt=1
- race=true
- run=1
- shard=12
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
/cc @cockroachlabs/kv-triage
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestCheckConsistencyInconsistent.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Contributor guide
Research direction
Start with kv/kvserver.TestCheckConsistencyInconsistent and reproduce the failure on the referenced release commit. Trace the checkpoint path through pkg/kv/kvserver/replica_consistency.go, pkg/kv/kvserver/store.go, and pkg/storage/pebble.go, using the open-file report in vfs/vfstest/open_files.go; done means the test passes without an open-file panic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100