cockroachdb / cockroachdb/cockroach
Sentry: crash_reporting.go:432: error during replicaGC: replica descriptor's ID has changed (2 >= 2) (1) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/util/log/logcra...
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.
Sentry Link: [https://cockroach-labs.sentry.io/issues/6973264118/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/6973264118/?referrer=webhooks_plugin)
Panic Message:
```
crash_reporting.go:432: error during replicaGC: replica descriptor's ID has changed (2 >= 2)
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/util/log/logcrash.ReportOrPanic
| pkg/util/log/logcrash/crash_reporting.go:432
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaGCQueue).process
| pkg/kv/kvserver/replica_gc_queue.go:323
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1
| pkg/kv/kvserver/queue.go:974
| github.com/cockroachdb/cockroach/pkg/util/timeutil.RunWithTimeout
| pkg/util/timeutil/timeout.go:28
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica
| pkg/kv/kvserver/queue.go:967
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processOneAsyncAndReleaseSem.func1
| pkg/kv/kvserver/queue.go:917
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
| pkg/util/stop/stopper.go:500
| runtime.goexit
| src/runtime/asm_amd64.s:1700
Wraps: (2) secondary error attachment
| replica descriptor's ID has changed (2 >= 2)
| (1) assertion failure
| Wraps: (2) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).removeInitializedReplicaRaftMuLocked.func1
| | pkg/kv/kvserver/store_remove_replica.go:131
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).removeInitializedReplicaRaftMuLocked
| | pkg/kv/kvserver/store_remove_replica.go:139
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).RemoveReplica
| | pkg/kv/kvserver/store_remove_replica.go:46
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaGCQueue).process
| | pkg/kv/kvserver/replica_gc_queue.go:318
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1
| | pkg/kv/kvserver/queue.go:974
| | github.com/cockroachdb/cockroach/pkg/util/timeutil.RunWithTimeout
| | pkg/util/timeutil/timeout.go:28
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica
| | pkg/kv/kvserver/queue.go:967
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processOneAsyncAndReleaseSem.func1
| | pkg/kv/kvserver/queue.go:917
| | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
| | pkg/util/stop/stopper.go:500
| | runtime.goexit
| | src/runtime/asm_amd64.s:1700
| Wraps: (3) replica descriptor's ID has changed (2 >= 2)
| Error types: (1) *assert.withAssertionFailure (2) *withstack.withStack (3) *errutil.leafError
Wraps: (3) error during replicaGC: replica descriptor's ID has changed (2 >= 2)
Error types: (1) *withstack.withStack (2) *secondary.withSecondaryError (3) *errutil.leafError
-- report composition:
*errutil.leafError: error during replicaGC: replica descriptor's ID has changed (2 >= 2)
*secondary.withSecondaryError: details for github.com/cockroachdb/errors/withstack/*withstack.withStack:::
crash_reporting.go:432: *withstack.withStack (top exception)
```
Stacktrace (expand for inline code snippets):
[src/runtime/asm_amd64.s#L1699-L1701](src/runtime/asm_amd64.s#L1699-L1701)
[pkg/util/stop/stopper.go#L499-L501](pkg/util/stop/stopper.go#L499-L501)
[pkg/kv/kvserver/queue.go#L916-L918](pkg/kv/kvserver/queue.go#L916-L918)
[pkg/kv/kvserver/queue.go#L966-L968](pkg/kv/kvserver/queue.go#L966-L968)
[pkg/util/timeutil/timeout.go#L27-L29](pkg/util/timeutil/timeout.go#L27-L29)
[pkg/kv/kvserver/queue.go#L973-L975](pkg/kv/kvserver/queue.go#L973-L975)
[pkg/kv/kvserver/replica_gc_queue.go#L322-L324](pkg/kv/kvserver/replica_gc_queue.go#L322-L324)
[pkg/util/log/logcrash/crash_reporting.go#L431-L433](pkg/util/log/logcrash/crash_reporting.go#L431-L433)
```
src/runtime/asm_amd64.s in runtime.goexit at line 1700
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 500
pkg/kv/kvserver/queue.go in pkg/kv/kvserver.(*baseQueue).processOneAsyncAndReleaseSem.func1 at line 917
pkg/kv/kvserver/queue.go in pkg/kv/kvserver.(*baseQueue).processReplica at line 967
pkg/util/timeutil/timeout.go in pkg/util/timeutil.RunWithTimeout at line 28
pkg/kv/kvserver/queue.go in pkg/kv/kvserver.(*baseQueue).processReplica.func1 at line 974
pkg/kv/kvserver/replica_gc_queue.go in pkg/kv/kvserver.(*replicaGCQueue).process at line 323
pkg/util/log/logcrash/crash_reporting.go in pkg/util/log/logcrash.ReportOrPanic at line 432
```
### Tags
| Tag | Value |
| --- | --- |
| Command | server |
| Environment | v25.2.2 |
| Go Version | go1.23.7 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v25.2.2 |
| Cockroach SHA | 0780d511bd527bfd9d02f4c7ca5ecc19115d9f5e |
| # of CPUs | 2 |
| # of Goroutines | 359 |
Jira issue: CRDB-55828
Contributor guide
Assessment
This issue has not been assessed yet.