pingcap / pingcap/tidb

Maximum concurrency control for a single SQL statement

Open
#67,769 0 comments 0 reactions 0 assignees View on GitHub
contribution report/customer sig/execution type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement
Currently, there are two variables controlling parallelism: `tidb_distsql_scan_concurrency` and `tidb_executor_concurrency`. In SQL queries involving multiple table joins, the maximum parallelism is the product of these two variables, with a default maximum of 75. Furthermore, in index join scenarios, this can easily lead to hotspots in a single region or a single tikv. A method is needed to control the maximum parallelism of a single SQL query.

Below is an example:
```
IndexJoin
TableScan
TableRowIDScan
```
If IndexJoin is using the default concurrency 5, and the default distsql_concurrency is 15, then the TableRowIDScan's distsql concurrency could be `5*15=75`?

If there is an IndexLookup like below, the TableRowIDScan's distsql concurrency could be `5*5*15 = 375`?
```
IndexJoin
TableRangeScan
IndexLoookup
IndexRowIDScan
TableRowIDScan
```

Should we set up a max concurrency for these SQLs?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.