pingcap / pingcap/tidb

coprocessor: do not ask unnecessary columns' info to avoid read values in TiKV if necessary

Open
#56,004 1 comment 0 reactions 0 assignees View on GitHub
component/executor report/customer type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement
```
drop table if exists t1, t2;

create table t1 (a int, b varchar(10), pad1 varchar(2048), pad2 varchar(2048), pad3 varchar(2048), primary key (a));
create table t2 (a int, b varchar(10), pad1 varchar(2048), pad2 varchar(2048), pad3 varchar(2048), primary key (b, a));

insert into t1 values (1, 1, repeat('a', 2048), repeat('b', 2048), repeat('c', 2048));
insert into t1 select a + 1, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 2, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 4, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 8, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 16, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 32, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 64, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 128, b, pad1, pad2, pad3 from t1;
insert into t2 select * from t1;

analyze table t1;
analyze table t2;

explain analyze select count(1) from t1;
explain analyze select count(1) from t2;
```
The two select count(*) statement should have the similar process key size.

```
Select from t1 ...total_process_keys: 256, total_process_keys_size: 6912...
Select from t2 ...total_process_keys: 256, total_process_keys_size: 1586944...
```
The t2 statement reads row value as well, so the total_process_keys_size is over hundred time of t1's.

Tikv determines whether to scan the value based on the columns_info in from the request. it will scan the value if it need any column from no-primary-key,related check is here:
https://github.com/tikv/tikv/blob/a4c0ea1657b3d939da51ea1cbbe77aff94bb60d3/components/tidb_query_executors/src/table_scan_executor.rs#L78-L83

Meanwhile, the column_info here comes from the request, the related proto are here:

https://github.com/pingcap/tipb/blob/e46e4632bd2b8c28a1a5f0986513bec8e25984e9/proto/executor.proto#L140-L154

```
message TableScan {
optional int64 table_id = 1 [(gogoproto.nullable) = false];
repeated ColumnInfo columns = 2;
optional bool desc = 3 [(gogoproto.nullable) = false];
repeated int64 primary_column_ids = 4;
optional EngineType next_read_engine = 5 [(gogoproto.nullable) = false]; // which engine we should in next step, only used by tiflash
repeated KeyRange ranges = 6 [(gogoproto.nullable) = false]; // For global read in join, we must point out the key ranges when we don't have the region info.
repeated int64 primary_prefix_column_ids = 7;
optional bool keep_order = 8;
optional bool is_fast_scan = 9; // fast_scan is a feature only provided by TiFlash (but not TiKV).
repeated Expr pushed_down_filter_conditions = 10; // conditions that are pushed down to storage layer, only used by TiFlash.
optional ANNQueryInfo ann_query = 13; // only used by TiFlash
repeated RuntimeFilter runtime_filter_list = 11; // only used by TiFlash
optional int32 max_wait_time_ms = 12 [(gogoproto.nullable) = false];
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.