simplify coprocessor goroutines to reduce latency in ad-hoc like queries
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
Now the coprocessor works with 3 types of goroutines.
* [`copIteratorWorker`](https://github.com/pingcap/tidb/blob/4b91fee6af2b1859d223670130e5c8bf3f400026/pkg/store/copr/coprocessor.go#L811-L841) handle the task, send it to stores.
* [`copIteratorTaskSender`](https://github.com/pingcap/tidb/blob/4b91fee6af2b1859d223670130e5c8bf3f400026/pkg/store/copr/coprocessor.go#L895-L929) dispatch the task and control the concurrency.
* [`Next`](https://github.com/pingcap/tidb/blob/4b91fee6af2b1859d223670130e5c8bf3f400026/pkg/store/copr/coprocessor.go#L1032-L1110) wait for data respoinse from worker.
```text
┌────────┐
│ │
┌───►│ worker ├───┐
│ │ │ │
│ └────────┘ │
│ │
│ ┌────────┐ │
│ │ │ │
├───►│ worker ├───┤
│ │ │ │
│ └────────┘ │
│ │
┌────────┐ │ ┌────────┐ │ ┌────────┐
│ │ │ │ │ │ │ │
│ sender ├───┼───►│ worker ├───┼───►│ Next │
│ │ │ │ │ │ │ │
└────────┘ │ └────────┘ │ └────────┘
│ │
│ ┌────────┐ │
│ │ │ │
├───►│ worker ├───┤
│ │ │ │
│ └────────┘ │
│ │
│ ┌────────┐ │
│ │ │ │
└───►│ worker ├───┘
│ │
└────────┘
```
When the number of tasks is low, the sender is not required, and it can just be:
```text
┌────────┐
│ │
│ worker ├───┐
│ │ │
└────────┘ │
│
┌────────┐ │
│ │ │
│ worker ├───┤
│ │ │
└────────┘ │
│
┌────────┐ │ ┌────────┐
│ │ │ │ │
│ worker ├───┼───►│ Next │
│ │ │ │ │
└────────┘ │ └────────┘
│
┌────────┐ │
│ │ │
│ worker ├───┤
│ │ │
└────────┘ │
│
┌────────┐ │
│ │ │
│ worker ├───┘
│ │
└────────┘
```
This can erase the sender goroutine and the channel between sender and workers.
Contributor guide
Assessment
This issue has not been assessed yet.