Sentry: txn_interceptor_seq_num_allocator.go:187: read sequence number 89 but sequence number is ignored [{30 30} {32 32} {34 34} {36 41} {43 43} {45 45} {81 81} {83 83} {85 89}] after savepoint rollb...

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: [https://cockroach-labs.sentry.io/issues/7674787090/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7674787090/?referrer=webhooks_plugin)

Panic Message:

```
txn_interceptor_seq_num_allocator.go:187: read sequence number 89 but sequence number is ignored [{30 30} {32 32} {34 34} {36 41} {43 43} {45 45} {81 81} {83 83} {85 89}] after savepoint rollback
(1) assertion failure
Wraps: (2)
| (opaque error wrapper)
| type name: github.com/cockroachdb/errors/withstack/*withstack.withStack
| reportable 0:
|
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).checkReadSeqNotIgnoredLocked
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go:187
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).SendLocked
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go:86
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnHeartbeater).SendLocked
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go:246
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*TxnCoordSender).Send
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/txn_coord_sender.go:537
| github.com/cockroachdb/cockroach/pkg/kv.(*DB).sendUsingSender
| github.com/cockroachdb/cockroach/pkg/kv/db.go:1138
| github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Send
| github.com/cockroachdb/cockroach/pkg/kv/txn.go:1282
| github.com/cockroachdb/cockroach/pkg/kv.sendAndFill
| github.com/cockroachdb/cockroach/pkg/kv/db.go:970
| github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Run
| github.com/cockroachdb/cockroach/pkg/kv/txn.go:800
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/catkv.catalogQuery.query
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/catkv/catalog_query.go:59
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/catkv.catalogReader.GetByIDs
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/catkv/catalog_reader.go:285
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/catkv.(*cachedCatalogReader).GetByIDs
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/catkv/catalog_reader_cached.go:352
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.getDescriptorsByID
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:207
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.ByIDGetter.Desc
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/getters.go:48
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.ByIDGetter.Table
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/getters.go:90
| github.com/cockroachdb/cockroach/pkg/sql.(*planner).LookupTableByID
| github.com/cockroachdb/cockroach/pkg/sql/planner.go:739
| github.com/cockroachdb/cockroach/pkg/sql.(*optCatalog).ResolveDataSourceByID
| github.com/cockroachdb/cockroach/pkg/sql/opt_catalog.go:333
| github.com/cockroachdb/cockroach/pkg/sql/opt.(*Metadata).CheckDependencies
| github.com/cockroachdb/cockroach/pkg/sql/opt/metadata.go:400
| github.com/cockroachdb/cockroach/pkg/sql/opt/memo.(*Memo).IsStale
| github.com/cockroachdb/cockroach/pkg/sql/opt/memo/memo.go:457
| github.com/cockroachdb/cockroach/pkg/sql.(*planner).prepareUsingOptimizer
| github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:147
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).populatePrepared
| github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:344
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).prepare.func2
| github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:284
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).prepare
| github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:289
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).addPreparedStmt
| github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:122
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execPrepare
| github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:83
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd
| github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2422
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run
| github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2213
| github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn
| github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:965
| github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommands
| github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:253
| github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).serveImpl.func3
| github.com/cockroachdb/cockroach/pkg/sql/pgwire/server.go:997
| runtime.goexit
| src/runtime/asm_amd64.s:1695
Wraps: (3) read sequence number 89 but sequence number is ignored [{30 30} {32 32} {34 34} {36 41} {43 43} {45 45} {81 81} {83 83} {85 89}] after savepoint rollback
Error types: (1) *assert.withAssertionFailure (2) *errbase.opaqueWrapper (3) *errutil.leafError
-- report composition:
*errutil.leafError: read sequence number 89 but sequence number is ignored [{30 30} {32 32} {34 34} {36 41} {43 43} {45 45} {81 81} {83 83} {85 89}] after savepoint rollback
txn_interceptor_seq_num_allocator.go:187: *withstack.withStack (top exception)
*assert.withAssertionFailure
```

Stacktrace (expand for inline code snippets):

[src/runtime/asm_amd64.s#L1694-L1696](src/runtime/asm_amd64.s#L1694-L1696)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/pgwire/server.go#L996-L998](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/pgwire/server.go#L996-L998)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/pgwire/conn.go#L252-L254](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/pgwire/conn.go#L252-L254)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor.go#L964-L966](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor.go#L964-L966)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor.go#L2212-L2214](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor.go#L2212-L2214)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor.go#L2421-L2423](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor.go#L2421-L2423)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L82-L84](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L82-L84)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L121-L123](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L121-L123)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L288-L290](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L288-L290)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L283-L285](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L283-L285)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L343-L345](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/conn_executor_prepare.go#L343-L345)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/plan_opt.go#L146-L148](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/plan_opt.go#L146-L148)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/opt/memo/memo.go#L456-L458](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/opt/memo/memo.go#L456-L458)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/opt/metadata.go#L399-L401](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/opt/metadata.go#L399-L401)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/opt_catalog.go#L332-L334](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/opt_catalog.go#L332-L334)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/planner.go#L738-L740](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/planner.go#L738-L740)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/descs/getters.go#L89-L91](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/descs/getters.go#L89-L91)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/descs/getters.go#L47-L49](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/descs/getters.go#L47-L49)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/descs/descriptor.go#L206-L208](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/descs/descriptor.go#L206-L208)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/internal/catkv/catalog_reader_cached.go#L351-L353](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/internal/catkv/catalog_reader_cached.go#L351-L353)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/internal/catkv/catalog_reader.go#L284-L286](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/internal/catkv/catalog_reader.go#L284-L286)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/internal/catkv/catalog_query.go#L58-L60](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/sql/catalog/internal/catkv/catalog_query.go#L58-L60)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/txn.go#L799-L801](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/txn.go#L799-L801)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/db.go#L969-L971](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/db.go#L969-L971)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/txn.go#L1281-L1283](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/txn.go#L1281-L1283)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/db.go#L1137-L1139](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/db.go#L1137-L1139)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/kvclient/kvcoord/txn_coord_sender.go#L536-L538](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/kvclient/kvcoord/txn_coord_sender.go#L536-L538)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go#L245-L247](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go#L245-L247)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go#L85-L87](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go#L85-L87)
[https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go#L186-L188](https://github.com/cockroachdb/cockroach/blob/484f8e2f4d3cb23582416944d79eab72ff42a6a1/pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go#L186-L188)

```
src/runtime/asm_amd64.s in runtime.goexit at line 1695
pkg/sql/pgwire/server.go in pkg/sql/pgwire.(*Server).serveImpl.func3 at line 997
pkg/sql/pgwire/conn.go in pkg/sql/pgwire.(*conn).processCommands at line 253
pkg/sql/conn_executor.go in pkg/sql.(*Server).ServeConn at line 965
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).run at line 2213
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd at line 2422
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).execPrepare at line 83
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).addPreparedStmt at line 122
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).prepare at line 289
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).prepare.func2 at line 284
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).populatePrepared at line 344
pkg/sql/plan_opt.go in pkg/sql.(*planner).prepareUsingOptimizer at line 147
pkg/sql/opt/memo/memo.go in pkg/sql/opt/memo.(*Memo).IsStale at line 457
pkg/sql/opt/metadata.go in pkg/sql/opt.(*Metadata).CheckDependencies at line 400
pkg/sql/opt_catalog.go in pkg/sql.(*optCatalog).ResolveDataSourceByID at line 333
pkg/sql/planner.go in pkg/sql.(*planner).LookupTableByID at line 739
pkg/sql/catalog/descs/getters.go in pkg/sql/catalog/descs.ByIDGetter.Table at line 90
pkg/sql/catalog/descs/getters.go in pkg/sql/catalog/descs.ByIDGetter.Desc at line 48
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.getDescriptorsByID at line 207
pkg/sql/catalog/internal/catkv/catalog_reader_cached.go in pkg/sql/catalog/internal/catkv.(*cachedCatalogReader).GetByIDs at line 352
pkg/sql/catalog/internal/catkv/catalog_reader.go in pkg/sql/catalog/internal/catkv.catalogReader.GetByIDs at line 285
pkg/sql/catalog/internal/catkv/catalog_query.go in pkg/sql/catalog/internal/catkv.catalogQuery.query at line 59
pkg/kv/txn.go in pkg/kv.(*Txn).Run at line 800
pkg/kv/db.go in pkg/kv.sendAndFill at line 970
pkg/kv/txn.go in pkg/kv.(*Txn).Send at line 1282
pkg/kv/db.go in pkg/kv.(*DB).sendUsingSender at line 1138
pkg/kv/kvclient/kvcoord/txn_coord_sender.go in pkg/kv/kvclient/kvcoord.(*TxnCoordSender).Send at line 537
pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go in pkg/kv/kvclient/kvcoord.(*txnHeartbeater).SendLocked at line 246
pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go in pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).SendLocked at line 86
pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go in pkg/kv/kvclient/kvcoord.(*txnSeqNumAllocator).checkReadSeqNotIgnoredLocked at line 187
```

### Tags

| Tag | Value |
| --- | --- |
| Command | start-single-node |
| Environment | v24.1.1 |
| Go Version | go1.22.3 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v24.1.1 |
| Cockroach SHA | 484f8e2f4d3cb23582416944d79eab72ff42a6a1 |
| # of CPUs | 20 |
| # of Goroutines | 591 |

Jira issue: CRDB-66732

Start in pkg/kv/kvclient/kvcoord/txn_interceptor_seq_num_allocator.go at checkReadSeqNotIgnoredLocked, line 187, then trace SendLocked and the savepoint rollback path shown in the stack trace. Follow the callers through txn_coord_sender.go, txn.go, and the catalog query and prepared-statement entry points. Done means the reported assertion is understood and resolved with regression coverage for the savepoint rollback sequence-number case.

Written by the indexing model from the issue text.