pingcap / pingcap/tidb

global sort merge-sort/write&ingest not limited by thread

Open
#50,222 4 comments 0 reactions 0 assignees View on GitHub
affects-8.1 component/ddl severity/moderate type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)
Import Into data using global sort with thread=16 vs thread=8

### 2. What did you expect to see? (Required)
Resource usage is limited by thread
### 3. What did you see instead (Required)
You can see that the resource utilization is the same for thread=8 and thread=16.
![img_v3_026t_2d60e884-be01-42f1-8e54-5b689f241ccg](https://github.com/pingcap/tidb/assets/10184437/6ac84bf2-2d7d-4903-990b-b2a31c036735)

### 4. What is your TiDB version? (Required)
```sql
[11:18:12]TiDB root:test> select tidb_version();
+-----------------------------------------------------------+
| tidb_version() |
+-----------------------------------------------------------+
| Release Version: v7.6.0-alpha |
| Edition: Community |
| Git Commit Hash: f0a4104fb03ee09c086d7a12144abee8d502117a |
| Git Branch: heads/refs/tags/v7.6.0-alpha |
| UTC Build Time: 2024-01-08 14:33:09 |
| GoVersion: go1.21.5 |
| Race Enabled: false |
| Check Table Before Drop: false |
| Store: tikv |
+-----------------------------------------------------------+
1 row in set
Time: 0.843s
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the global-sort import with thread=8 and thread=16, then trace the merge-sort/write&ingest path responsible for resource usage. Done means resource utilization is constrained by the configured thread count and the two runs no longer use the same resources.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.