pingcap / pingcap/tidb

Seems that IndexLookUpReader may generate unexpected unordered kv ranges for its table scan

Open
#64,368 2 comments 0 reactions 0 assignees View on GitHub
may-affects-6.5 may-affects-7.1 may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/major sig/execution type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

Unknown yet.

### 2. What did you expect to see? (Required)

### 3. What did you see instead (Required)

We saw an error stack
```
LocateBucket returned nil: startKey outside location boundaries. This indicates either: (1) PD returned inconsistent metadata, (2) our range splitting logic has a bug, or (3) upstream LocationKeyRanges was constructed incorrectly. See error log above for full diagnostics.
github.com/pingcap/tidb/pkg/util.GetRecoverError
/workspace/source/tidb/pkg/util/util.go:317
github.com/pingcap/tidb/pkg/executor.execTableTask.func1
/workspace/source/tidb/pkg/executor/distsql.go:1487
runtime.gopanic
/usr/local/go/src/runtime/panic.go:791
github.com/pingcap/tidb/pkg/store/copr.(*RegionCache).SplitKeyRangesByBuckets.func1
/workspace/source/tidb/pkg/store/copr/region_cache.go:696
runtime.gopanic
/usr/local/go/src/runtime/panic.go:791
github.com/pingcap/tidb/pkg/store/copr.(*LocationKeyRanges).splitKeyRangesByBuckets
/workspace/source/tidb/pkg/store/copr/region_cache.go:441
github.com/pingcap/tidb/pkg/store/copr.(*RegionCache).SplitKeyRangesByBuckets
/workspace/source/tidb/pkg/store/copr/region_cache.go:707
github.com/pingcap/tidb/pkg/store/copr.buildCopTasks
/workspace/source/tidb/pkg/store/copr/coprocessor.go:366
github.com/pingcap/tidb/pkg/store/copr.(*CopClient).BuildCopIterator.func3
/workspace/source/tidb/pkg/store/copr/coprocessor.go:158
github.com/pingcap/tidb/pkg/kv.(*KeyRanges).ForEachPartitionWithErr
/workspace/source/tidb/pkg/kv/kv.go:481
github.com/pingcap/tidb/pkg/store/copr.(*CopClient).BuildCopIterator
/workspace/source/tidb/pkg/store/copr/coprocessor.go:172
github.com/pingcap/tidb/pkg/store/copr.(*CopClient).Send
/workspace/source/tidb/pkg/store/copr/coprocessor.go:102
github.com/pingcap/tidb/pkg/distsql.Select
/workspace/source/tidb/pkg/distsql/distsql.go:109
github.com/pingcap/tidb/pkg/distsql.SelectWithRuntimeStats
/workspace/source/tidb/pkg/distsql/distsql.go:165
github.com/pingcap/tidb/pkg/executor.selectResultHook.SelectResult
/workspace/source/tidb/pkg/executor/table_reader.go:73
github.com/pingcap/tidb/pkg/executor.(*dataReaderBuilder).buildTableReaderBase
/workspace/source/tidb/pkg/executor/builder.go:5049
github.com/pingcap/tidb/pkg/executor.(*dataReaderBuilder).buildTableReaderFromHandles
/workspace/source/tidb/pkg/executor/builder.go:5073
github.com/pingcap/tidb/pkg/executor.(*IndexLookUpExecutor).buildTableReader
/workspace/source/tidb/pkg/executor/distsql.go:977
github.com/pingcap/tidb/pkg/executor.(*tableWorker).executeTask
/workspace/source/tidb/pkg/executor/distsql.go:1786
github.com/pingcap/tidb/pkg/executor.execTableTask
/workspace/source/tidb/pkg/executor/distsql.go:1505
github.com/pingcap/tidb/pkg/executor.(*indexWorker).fetchHandles.func2
/workspace/source/tidb/pkg/executor/distsql.go:1285
github.com/pingcap/tidb/pkg/executor.(*workerPool).run
/workspace/source/tidb/pkg/executor/utils.go:239
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1700
```

We haven't known why the panic occurred.
But it's clear that the wrong key ranges are built from the IndexLookUpReader's Table Scan from stack: `github.com/pingcap/tidb/pkg/executor.(*IndexLookUpExecutor).buildTableReader`

### 4. What is your TiDB version? (Required)
current master

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.