cockroachdb / cockroachdb/cockroach
Sentry: panic.go:770: runtime error: slice bounds out of range [336:0] (1) attached stack trace -- stack trace: | runtime.gopanic | GOROOT/src/runtime/panic.go:770 | github.com/cockroachdb/co...
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.
Sentry Link: [https://cockroach-labs.sentry.io/issues/7286835802/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7286835802/?referrer=webhooks_plugin)
Panic Message:
```
panic.go:770: runtime error: slice bounds out of range [336:0]
(1) attached stack trace
-- stack trace:
| runtime.gopanic
| GOROOT/src/runtime/panic.go:770
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).SendWithWriteBytes.func1
| pkg/kv/kvserver/store_send.go:104
| runtime.gopanic
| GOROOT/src/runtime/panic.go:770
| runtime.goPanicSliceB
| GOROOT/src/runtime/panic.go:154
| github.com/cockroachdb/pebble/sstable/block.Value.BlockData
| external/com_github_cockroachdb_pebble/sstable/block/buffer_pool.go:44
| github.com/cockroachdb/pebble/sstable/block.(*Reader).doRead
| external/com_github_cockroachdb_pebble/sstable/block/block.go:507
| github.com/cockroachdb/pebble/sstable/block.(*Reader).Read
| external/com_github_cockroachdb_pebble/sstable/block/block.go:459
| github.com/cockroachdb/pebble/sstable.(*Reader).readDataBlock
| external/com_github_cockroachdb_pebble/sstable/reader.go:336
| github.com/cockroachdb/pebble/sstable.(*singleLevelIterator[...]).loadDataBlock
| external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go:528
| github.com/cockroachdb/pebble/sstable.(*singleLevelIterator[...]).seekGEHelper
| external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go:787
| github.com/cockroachdb/pebble/sstable.(*singleLevelIterator[...]).SeekGE
| external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go:713
| github.com/cockroachdb/pebble.(*levelIter).SeekGE
| external/com_github_cockroachdb_pebble/level_iter.go:658
| github.com/cockroachdb/pebble.(*mergingIter).seekGE
| external/com_github_cockroachdb_pebble/merging_iter.go:978
| github.com/cockroachdb/pebble.(*mergingIter).SeekGE
| external/com_github_cockroachdb_pebble/merging_iter.go:1026
| github.com/cockroachdb/pebble.(*lazyCombinedIter).SeekGE
| external/com_github_cockroachdb_pebble/range_keys.go:606
| github.com/cockroachdb/pebble.(*Iterator).SeekGEWithLimit
| external/com_github_cockroachdb_pebble/iterator.go:1348
| github.com/cockroachdb/pebble.(*Iterator).SeekGE
| external/com_github_cockroachdb_pebble/iterator.go:1252
| github.com/cockroachdb/cockroach/pkg/storage.(*pebbleIterator).SeekGE
| pkg/storage/pebble_iterator.go:351
| github.com/cockroachdb/cockroach/pkg/storage.(*pebbleMVCCScanner).iterSeek
| pkg/storage/pebble_mvcc_scanner.go:1722
| github.com/cockroachdb/cockroach/pkg/storage.(*pebbleMVCCScanner).seekToStartOfScan
| pkg/storage/pebble_mvcc_scanner.go:634
| github.com/cockroachdb/cockroach/pkg/storage.(*pebbleMVCCScanner).scan
| pkg/storage/pebble_mvcc_scanner.go:680
| github.com/cockroachdb/cockroach/pkg/storage.mvccScanToBytes
| pkg/storage/mvcc.go:4603
| github.com/cockroachdb/cockroach/pkg/storage.MVCCScanToBytes
| pkg/storage/mvcc.go:4959
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval.Scan
| pkg/kv/kvserver/batcheval/cmd_scan.go:71
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.evaluateCommand
| pkg/kv/kvserver/replica_evaluate.go:487
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.evaluateBatch
| pkg/kv/kvserver/replica_evaluate.go:304
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeReadOnlyBatchWithServersideRefreshes
| pkg/kv/kvserver/replica_read.go:471
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeReadOnlyBatch
| pkg/kv/kvserver/replica_read.go:131
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeBatchWithConcurrencyRetries
| pkg/kv/kvserver/replica_send.go:498
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).SendWithWriteBytes
| pkg/kv/kvserver/replica_send.go:173
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).SendWithWriteBytes
| pkg/kv/kvserver/store_send.go:188
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).SendWithWriteBytes
| pkg/kv/kvserver/stores.go:199
Wraps: (2) runtime error: slice bounds out of range [336:0]
Error types: (1) *withstack.withStack (2) runtime.boundsError
-- report composition:
runtime.boundsError
panic.go:770: *withstack.withStack (top exception)
```
Stacktrace (expand for inline code snippets):
[pkg/kv/kvserver/stores.go#L198-L200](pkg/kv/kvserver/stores.go#L198-L200)
[pkg/kv/kvserver/store_send.go#L187-L189](pkg/kv/kvserver/store_send.go#L187-L189)
[pkg/kv/kvserver/replica_send.go#L172-L174](pkg/kv/kvserver/replica_send.go#L172-L174)
[pkg/kv/kvserver/replica_send.go#L497-L499](pkg/kv/kvserver/replica_send.go#L497-L499)
[pkg/kv/kvserver/replica_read.go#L130-L132](pkg/kv/kvserver/replica_read.go#L130-L132)
[pkg/kv/kvserver/replica_read.go#L470-L472](pkg/kv/kvserver/replica_read.go#L470-L472)
[pkg/kv/kvserver/replica_evaluate.go#L303-L305](pkg/kv/kvserver/replica_evaluate.go#L303-L305)
[pkg/kv/kvserver/replica_evaluate.go#L486-L488](pkg/kv/kvserver/replica_evaluate.go#L486-L488)
[pkg/kv/kvserver/batcheval/cmd_scan.go#L70-L72](pkg/kv/kvserver/batcheval/cmd_scan.go#L70-L72)
[pkg/storage/mvcc.go#L4958-L4960](pkg/storage/mvcc.go#L4958-L4960)
[pkg/storage/mvcc.go#L4602-L4604](pkg/storage/mvcc.go#L4602-L4604)
[pkg/storage/pebble_mvcc_scanner.go#L679-L681](pkg/storage/pebble_mvcc_scanner.go#L679-L681)
[pkg/storage/pebble_mvcc_scanner.go#L633-L635](pkg/storage/pebble_mvcc_scanner.go#L633-L635)
[pkg/storage/pebble_mvcc_scanner.go#L1721-L1723](pkg/storage/pebble_mvcc_scanner.go#L1721-L1723)
[pkg/storage/pebble_iterator.go#L350-L352](pkg/storage/pebble_iterator.go#L350-L352)
[external/com_github_cockroachdb_pebble/iterator.go#L1251-L1253](external/com_github_cockroachdb_pebble/iterator.go#L1251-L1253)
[external/com_github_cockroachdb_pebble/iterator.go#L1347-L1349](external/com_github_cockroachdb_pebble/iterator.go#L1347-L1349)
[external/com_github_cockroachdb_pebble/range_keys.go#L605-L607](external/com_github_cockroachdb_pebble/range_keys.go#L605-L607)
[external/com_github_cockroachdb_pebble/merging_iter.go#L1025-L1027](external/com_github_cockroachdb_pebble/merging_iter.go#L1025-L1027)
[external/com_github_cockroachdb_pebble/merging_iter.go#L977-L979](external/com_github_cockroachdb_pebble/merging_iter.go#L977-L979)
[external/com_github_cockroachdb_pebble/level_iter.go#L657-L659](external/com_github_cockroachdb_pebble/level_iter.go#L657-L659)
[external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go#L712-L714](external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go#L712-L714)
[external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go#L786-L788](external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go#L786-L788)
[external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go#L527-L529](external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go#L527-L529)
[external/com_github_cockroachdb_pebble/sstable/reader.go#L335-L337](external/com_github_cockroachdb_pebble/sstable/reader.go#L335-L337)
[external/com_github_cockroachdb_pebble/sstable/block/block.go#L458-L460](external/com_github_cockroachdb_pebble/sstable/block/block.go#L458-L460)
[external/com_github_cockroachdb_pebble/sstable/block/block.go#L506-L508](external/com_github_cockroachdb_pebble/sstable/block/block.go#L506-L508)
[external/com_github_cockroachdb_pebble/sstable/block/buffer_pool.go#L43-L45](external/com_github_cockroachdb_pebble/sstable/block/buffer_pool.go#L43-L45)
[GOROOT/src/runtime/panic.go#L153-L155](GOROOT/src/runtime/panic.go#L153-L155)
[GOROOT/src/runtime/panic.go#L769-L771](GOROOT/src/runtime/panic.go#L769-L771)
[pkg/kv/kvserver/store_send.go#L103-L105](pkg/kv/kvserver/store_send.go#L103-L105)
[GOROOT/src/runtime/panic.go#L769-L771](GOROOT/src/runtime/panic.go#L769-L771)
```
pkg/kv/kvserver/stores.go in pkg/kv/kvserver.(*Stores).SendWithWriteBytes at line 199
pkg/kv/kvserver/store_send.go in pkg/kv/kvserver.(*Store).SendWithWriteBytes at line 188
pkg/kv/kvserver/replica_send.go in pkg/kv/kvserver.(*Replica).SendWithWriteBytes at line 173
pkg/kv/kvserver/replica_send.go in pkg/kv/kvserver.(*Replica).executeBatchWithConcurrencyRetries at line 498
pkg/kv/kvserver/replica_read.go in pkg/kv/kvserver.(*Replica).executeReadOnlyBatch at line 131
pkg/kv/kvserver/replica_read.go in pkg/kv/kvserver.(*Replica).executeReadOnlyBatchWithServersideRefreshes at line 471
pkg/kv/kvserver/replica_evaluate.go in pkg/kv/kvserver.evaluateBatch at line 304
pkg/kv/kvserver/replica_evaluate.go in pkg/kv/kvserver.evaluateCommand at line 487
pkg/kv/kvserver/batcheval/cmd_scan.go in pkg/kv/kvserver/batcheval.Scan at line 71
pkg/storage/mvcc.go in pkg/storage.MVCCScanToBytes at line 4959
pkg/storage/mvcc.go in pkg/storage.mvccScanToBytes at line 4603
pkg/storage/pebble_mvcc_scanner.go in pkg/storage.(*pebbleMVCCScanner).scan at line 680
pkg/storage/pebble_mvcc_scanner.go in pkg/storage.(*pebbleMVCCScanner).seekToStartOfScan at line 634
pkg/storage/pebble_mvcc_scanner.go in pkg/storage.(*pebbleMVCCScanner).iterSeek at line 1722
pkg/storage/pebble_iterator.go in pkg/storage.(*pebbleIterator).SeekGE at line 351
external/com_github_cockroachdb_pebble/iterator.go in github.com/cockroachdb/pebble.(*Iterator).SeekGE at line 1252
external/com_github_cockroachdb_pebble/iterator.go in github.com/cockroachdb/pebble.(*Iterator).SeekGEWithLimit at line 1348
external/com_github_cockroachdb_pebble/range_keys.go in github.com/cockroachdb/pebble.(*lazyCombinedIter).SeekGE at line 606
external/com_github_cockroachdb_pebble/merging_iter.go in github.com/cockroachdb/pebble.(*mergingIter).SeekGE at line 1026
external/com_github_cockroachdb_pebble/merging_iter.go in github.com/cockroachdb/pebble.(*mergingIter).seekGE at line 978
external/com_github_cockroachdb_pebble/level_iter.go in github.com/cockroachdb/pebble.(*levelIter).SeekGE at line 658
external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go in github.com/cockroachdb/pebble/sstable.(*singleLevelIterator[...]).SeekGE at line 713
external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go in github.com/cockroachdb/pebble/sstable.(*singleLevelIterator[...]).seekGEHelper at line 787
external/com_github_cockroachdb_pebble/sstable/reader_iter_single_lvl.go in github.com/cockroachdb/pebble/sstable.(*singleLevelIterator[...]).loadDataBlock at line 528
external/com_github_cockroachdb_pebble/sstable/reader.go in github.com/cockroachdb/pebble/sstable.(*Reader).readDataBlock at line 336
external/com_github_cockroachdb_pebble/sstable/block/block.go in github.com/cockroachdb/pebble/sstable/block.(*Reader).Read at line 459
external/com_github_cockroachdb_pebble/sstable/block/block.go in github.com/cockroachdb/pebble/sstable/block.(*Reader).doRead at line 507
external/com_github_cockroachdb_pebble/sstable/block/buffer_pool.go in github.com/cockroachdb/pebble/sstable/block.Value.BlockData at line 44
GOROOT/src/runtime/panic.go in runtime.goPanicSliceB at line 154
GOROOT/src/runtime/panic.go in runtime.gopanic at line 770
pkg/kv/kvserver/store_send.go in pkg/kv/kvserver.(*Store).SendWithWriteBytes.func1 at line 104
GOROOT/src/runtime/panic.go in runtime.gopanic at line 770
```
### Tags
| Tag | Value |
| --- | --- |
| Command | server |
| Environment | v25.1.5 |
| Go Version | go1.22.8 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v25.1.5 |
| Cockroach SHA | d148d3abe6cb84ecc15c470c524f05c1e726c671 |
| # of CPUs | 8 |
| # of Goroutines | 540 |
Jira issue: CRDB-60683
Contributor guide
Research direction
Start at external/com_github_cockroachdb_pebble/sstable/block/buffer_pool.go:44 and trace the reported call chain through pkg/storage/pebble_mvcc_scanner.go and pkg/kv/kvserver/batcheval/cmd_scan.go. Determine the cause of the slice-bounds panic in the v25.1.5 scan path; done means the crash is prevented and the relevant scan behavior is verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 22/100