pingcap / pingcap/tidb

simplify coprocessor goroutines to reduce latency in ad-hoc like queries

Open
#53,250 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

Now the coprocessor works with 3 types of goroutines.

* [`copIteratorWorker`](https://github.com/pingcap/tidb/blob/4b91fee6af2b1859d223670130e5c8bf3f400026/pkg/store/copr/coprocessor.go#L811-L841) handle the task, send it to stores.
* [`copIteratorTaskSender`](https://github.com/pingcap/tidb/blob/4b91fee6af2b1859d223670130e5c8bf3f400026/pkg/store/copr/coprocessor.go#L895-L929) dispatch the task and control the concurrency.
* [`Next`](https://github.com/pingcap/tidb/blob/4b91fee6af2b1859d223670130e5c8bf3f400026/pkg/store/copr/coprocessor.go#L1032-L1110) wait for data respoinse from worker.

```text
┌────────┐
│ │
┌───►│ worker ├───┐
│ │ │ │
│ └────────┘ │
│ │
│ ┌────────┐ │
│ │ │ │
├───►│ worker ├───┤
│ │ │ │
│ └────────┘ │
│ │
┌────────┐ │ ┌────────┐ │ ┌────────┐
│ │ │ │ │ │ │ │
│ sender ├───┼───►│ worker ├───┼───►│ Next │
│ │ │ │ │ │ │ │
└────────┘ │ └────────┘ │ └────────┘
│ │
│ ┌────────┐ │
│ │ │ │
├───►│ worker ├───┤
│ │ │ │
│ └────────┘ │
│ │
│ ┌────────┐ │
│ │ │ │
└───►│ worker ├───┘
│ │
└────────┘
```

When the number of tasks is low, the sender is not required, and it can just be:

```text
┌────────┐
│ │
│ worker ├───┐
│ │ │
└────────┘ │

┌────────┐ │
│ │ │
│ worker ├───┤
│ │ │
└────────┘ │

┌────────┐ │ ┌────────┐
│ │ │ │ │
│ worker ├───┼───►│ Next │
│ │ │ │ │
└────────┘ │ └────────┘

┌────────┐ │
│ │ │
│ worker ├───┤
│ │ │
└────────┘ │

┌────────┐ │
│ │ │
│ worker ├───┘
│ │
└────────┘
```

This can erase the sender goroutine and the channel between sender and workers.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.