cockroachdb / cockroachdb/cockroach

Sentry: txn_interceptor_seq_num_allocator.go:165: cannot stepReadSeqLocked() after mistaken initialization (-1794692256,0) (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | g...

Open
#153,478 1 comment 0 reactions 0 assignees View on GitHub
branch-release-24.3 C-bug O-sentry T-kv X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: [https://cockroach-labs.sentry.io/issues/6875425851/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/6875425851/?referrer=webhooks_plugin)

Panic Message:

```
txn_interceptor_seq_num_allocator.go:165: cannot stepReadSeqLocked() after mistaken initialization (-1794692256,0)
(1) assertion failure
Wraps: (2) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).stepReadSeqLocked
| pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go:165
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).manualStepReadSeqLocked
| pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go:147
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*TxnCoordSender).Step
| pkg/kv/kvclient/kvcoord/txn_coord_sender.go:1451
| github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Step
| pkg/kv/txn.go:1695
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).commitSQLTransactionInternal
| pkg/sql/conn_executor_exec.go:1584
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).commitSQLTransaction
| pkg/sql/conn_executor_exec.go:1498
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
| pkg/sql/conn_executor_exec.go:916
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt.func1
| pkg/sql/conn_executor_exec.go:141
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithProfiling
| pkg/sql/conn_executor_exec.go:3451
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt
| pkg/sql/conn_executor_exec.go:140
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1
| pkg/sql/conn_executor.go:2341
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd
| pkg/sql/conn_executor.go:2346
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run
| pkg/sql/conn_executor.go:2263
| github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn
| pkg/sql/conn_executor.go:963
| github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommands
| pkg/sql/pgwire/conn.go:251
| github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).serveImpl.func4
| pkg/sql/pgwire/server.go:1196
| runtime.goexit
| src/runtime/asm_amd64.s:1695
Wraps: (3) cannot stepReadSeqLocked() after mistaken initialization (-1794692256,0)
Error types: (1) *assert.withAssertionFailure (2) *withstack.withStack (3) *errutil.leafError
-- report composition:
*errutil.leafError: cannot stepReadSeqLocked() after mistaken initialization (-1794692256,0)
txn_interceptor_seq_num_allocator.go:165: *withstack.withStack (top exception)
*assert.withAssertionFailure
```

Stacktrace (expand for inline code snippets):

[src/runtime/asm_amd64.s#L1694-L1696](src/runtime/asm_amd64.s#L1694-L1696)
[pkg/sql/pgwire/server.go#L1195-L1197](pkg/sql/pgwire/server.go#L1195-L1197)
[pkg/sql/pgwire/conn.go#L250-L252](pkg/sql/pgwire/conn.go#L250-L252)
[pkg/sql/conn_executor.go#L962-L964](pkg/sql/conn_executor.go#L962-L964)
[pkg/sql/conn_executor.go#L2262-L2264](pkg/sql/conn_executor.go#L2262-L2264)
[pkg/sql/conn_executor.go#L2345-L2347](pkg/sql/conn_executor.go#L2345-L2347)
[pkg/sql/conn_executor.go#L2340-L2342](pkg/sql/conn_executor.go#L2340-L2342)
[pkg/sql/conn_executor_exec.go#L139-L141](pkg/sql/conn_executor_exec.go#L139-L141)
[pkg/sql/conn_executor_exec.go#L3450-L3452](pkg/sql/conn_executor_exec.go#L3450-L3452)
[pkg/sql/conn_executor_exec.go#L140-L142](pkg/sql/conn_executor_exec.go#L140-L142)
[pkg/sql/conn_executor_exec.go#L915-L917](pkg/sql/conn_executor_exec.go#L915-L917)
[pkg/sql/conn_executor_exec.go#L1497-L1499](pkg/sql/conn_executor_exec.go#L1497-L1499)
[pkg/sql/conn_executor_exec.go#L1583-L1585](pkg/sql/conn_executor_exec.go#L1583-L1585)
[pkg/kv/txn.go#L1694-L1696](pkg/kv/txn.go#L1694-L1696)
[pkg/kv/kvclient/kvcoord/txn_coord_sender.go#L1450-L1452](pkg/kv/kvclient/kvcoord/txn_coord_sender.go#L1450-L1452)
[pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go#L146-L148](pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go#L146-L148)
[pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go#L164-L166](pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go#L164-L166)

```
src/runtime/asm_amd64.s in runtime.goexit at line 1695
pkg/sql/pgwire/server.go in pkg/sql/pgwire.(*Server).serveImpl.func4 at line 1196
pkg/sql/pgwire/conn.go in pkg/sql/pgwire.(*conn).processCommands at line 251
pkg/sql/conn_executor.go in pkg/sql.(*Server).ServeConn at line 963
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).run at line 2263
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd at line 2346
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd.func1 at line 2341
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt at line 140
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithProfiling at line 3451
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt.func1 at line 141
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmtInOpenState at line 916
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).commitSQLTransaction at line 1498
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).commitSQLTransactionInternal at line 1584
pkg/kv/txn.go in pkg/kv.(*Txn).Step at line 1695
pkg/kv/kvclient/kvcoord/txn_coord_sender.go in pkg/kv/kvclient/kvcoord.(*TxnCoordSender).Step at line 1451
pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go in pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).manualStepReadSeqLocked at line 147
pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go in pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).stepReadSeqLocked at line 165
```

### Tags

| Tag | Value |
| --- | --- |
| Command | server |
| Environment | v24.3.2 |
| Go Version | go1.22.8 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v24.3.2 |
| Cockroach SHA | 2c5477e5ba1a433059b1d36bb725a6a89c840092 |
| # of CPUs | 8 |
| # of Goroutines | 883 |

Jira issue: CRDB-54442

Contributor guide

Open the contributing guide

Research direction

Start with pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go around lines 147 and 165, then trace the callers in txn_coord_sender.go and pkg/kv/txn.go shown in the stack trace. Determine the initialization and stepping sequence that produces the assertion, and establish a reproducible regression case whose completed behavior no longer reports this crash.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.