cockroachdb / cockroachdb/cockroach
Sentry: resolved_timestamp.go:292: log.Fatal: resolved timestamp 1781216885.337464928,0 equal to or above timestamp of operation × (1) attached stack trace -- stack trace: | github.com/cockroachd...
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.
Sentry Link: [https://cockroach-labs.sentry.io/issues/7545113185/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7545113185/?referrer=webhooks_plugin)
Panic Message:
```
resolved_timestamp.go:292: log.Fatal: resolved timestamp 1781216885.337464928,0 equal to or above timestamp of operation ×
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).assertOpAboveRTS
| pkg/kv/kvserver/rangefeed/resolved_timestamp.go:292
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).consumeLogicalOp
| pkg/kv/kvserver/rangefeed/resolved_timestamp.go:160
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).ConsumeLogicalOp
| pkg/kv/kvserver/rangefeed/resolved_timestamp.go:140
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).consumeLogicalOps
| pkg/kv/kvserver/rangefeed/scheduled_processor.go:701
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).consumeEvent
| pkg/kv/kvserver/rangefeed/scheduled_processor.go:639
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).processEvents
| pkg/kv/kvserver/rangefeed/scheduled_processor.go:175
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).process
| pkg/kv/kvserver/rangefeed/scheduled_processor.go:140
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*schedulerShard).processEvents
| pkg/kv/kvserver/rangefeed/scheduler.go:492
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Scheduler).Start.func1
| pkg/kv/kvserver/rangefeed/scheduler.go:249
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
| pkg/util/stop/stopper.go:498
| runtime.goexit
| src/runtime/asm_amd64.s:1695
Wraps: (2) secondary error attachment
| resolved timestamp 1781216885.337464928,0 equal to or above timestamp of operation ×
| (1) assertion failure
| Wraps: (2) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).assertOpAboveRTS
| | pkg/kv/kvserver/rangefeed/resolved_timestamp.go:289
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).consumeLogicalOp
| | pkg/kv/kvserver/rangefeed/resolved_timestamp.go:160
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).ConsumeLogicalOp
| | pkg/kv/kvserver/rangefeed/resolved_timestamp.go:140
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).consumeLogicalOps
| | pkg/kv/kvserver/rangefeed/scheduled_processor.go:701
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).consumeEvent
| | pkg/kv/kvserver/rangefeed/scheduled_processor.go:639
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).processEvents
| | pkg/kv/kvserver/rangefeed/scheduled_processor.go:175
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).process
| | pkg/kv/kvserver/rangefeed/scheduled_processor.go:140
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*schedulerShard).processEvents
| | pkg/kv/kvserver/rangefeed/scheduler.go:492
| | github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Scheduler).Start.func1
| | pkg/kv/kvserver/rangefeed/scheduler.go:249
| | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
| | pkg/util/stop/stopper.go:498
| | runtime.goexit
| | src/runtime/asm_amd64.s:1695
| Wraps: (3) resolved timestamp 1781216885.337464928,0 equal to or above timestamp of operation ×
| Error types: (1) *assert.withAssertionFailure (2) *withstack.withStack (3) *errutil.leafError
Wraps: (3) log.Fatal: resolved timestamp 1781216885.337464928,0 equal to or above timestamp of operation ×
Error types: (1) *withstack.withStack (2) *secondary.withSecondaryError (3) *errutil.leafError
-- report composition:
*errutil.leafError: log.Fatal: resolved timestamp 1781216885.337464928,0 equal to or above timestamp of operation ×
*secondary.withSecondaryError: details for github.com/cockroachdb/errors/withstack/*withstack.withStack:::
resolved_timestamp.go:292: *withstack.withStack (top exception)
```
Stacktrace (expand for inline code snippets):
[src/runtime/asm_amd64.s#L1694-L1696](src/runtime/asm_amd64.s#L1694-L1696)
[pkg/util/stop/stopper.go#L497-L499](pkg/util/stop/stopper.go#L497-L499)
[pkg/kv/kvserver/rangefeed/scheduler.go#L248-L250](pkg/kv/kvserver/rangefeed/scheduler.go#L248-L250)
[pkg/kv/kvserver/rangefeed/scheduler.go#L491-L493](pkg/kv/kvserver/rangefeed/scheduler.go#L491-L493)
[pkg/kv/kvserver/rangefeed/scheduled_processor.go#L139-L141](pkg/kv/kvserver/rangefeed/scheduled_processor.go#L139-L141)
[pkg/kv/kvserver/rangefeed/scheduled_processor.go#L174-L176](pkg/kv/kvserver/rangefeed/scheduled_processor.go#L174-L176)
[pkg/kv/kvserver/rangefeed/scheduled_processor.go#L638-L640](pkg/kv/kvserver/rangefeed/scheduled_processor.go#L638-L640)
[pkg/kv/kvserver/rangefeed/scheduled_processor.go#L700-L702](pkg/kv/kvserver/rangefeed/scheduled_processor.go#L700-L702)
[pkg/kv/kvserver/rangefeed/resolved_timestamp.go#L139-L141](pkg/kv/kvserver/rangefeed/resolved_timestamp.go#L139-L141)
[pkg/kv/kvserver/rangefeed/resolved_timestamp.go#L159-L161](pkg/kv/kvserver/rangefeed/resolved_timestamp.go#L159-L161)
[pkg/kv/kvserver/rangefeed/resolved_timestamp.go#L291-L293](pkg/kv/kvserver/rangefeed/resolved_timestamp.go#L291-L293)
```
src/runtime/asm_amd64.s in runtime.goexit at line 1695
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 498
pkg/kv/kvserver/rangefeed/scheduler.go in pkg/kv/kvserver/rangefeed.(*Scheduler).Start.func1 at line 249
pkg/kv/kvserver/rangefeed/scheduler.go in pkg/kv/kvserver/rangefeed.(*schedulerShard).processEvents at line 492
pkg/kv/kvserver/rangefeed/scheduled_processor.go in pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).process at line 140
pkg/kv/kvserver/rangefeed/scheduled_processor.go in pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).processEvents at line 175
pkg/kv/kvserver/rangefeed/scheduled_processor.go in pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).consumeEvent at line 639
pkg/kv/kvserver/rangefeed/scheduled_processor.go in pkg/kv/kvserver/rangefeed.(*ScheduledProcessor).consumeLogicalOps at line 701
pkg/kv/kvserver/rangefeed/resolved_timestamp.go in pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).ConsumeLogicalOp at line 140
pkg/kv/kvserver/rangefeed/resolved_timestamp.go in pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).consumeLogicalOp at line 160
pkg/kv/kvserver/rangefeed/resolved_timestamp.go in pkg/kv/kvserver/rangefeed.(*resolvedTimestamp).assertOpAboveRTS at line 292
```
### Tags
| Tag | Value |
| --- | --- |
| Command | start-single-node |
| Environment | v24.3.17 |
| Go Version | go1.22.8 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v24.3.17 |
| Cockroach SHA | 4b7d71c98252df426bd10a14b1a580c48de1af8c |
| # of CPUs | 32 |
| # of Goroutines | 648 |
Jira issue: CRDB-64706
Contributor guide
Assessment
This issue has not been assessed yet.