pingcap / pingcap/tidb

Performance problem for localColumnPool

Open
#48,314 0 comments 0 reactions 0 assignees View on GitHub
severity/moderate sig/execution type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

Add a test in pkg expression.
```
func TestTTTT(t *testing.T) {
cases := []int{1, 2, 4, 8}
tm := time.Now()
for _, n := range cases {
wg := sync.WaitGroup{}
wg.Add(n)
for i := 0; i < n; i++ {
go func() {
defer wg.Done()
for i := 0; i <= 1000000; i++ {
pool := newLocalColumnPool()
_ = pool.Get()
}

}()
}
wg.Wait()
fmt.Println(time.Since(tm))
}
}
```
And the the test result:
=== RUN TestTTTT
980.168366ms
2.931334393s
7.176344671s
15.385955085s
--- PASS: TestTTTT (15.39s)

If I alloc 5 columns for each loop, it is faster.
```
func TestTTTT(t *testing.T) {
cases := []int{1, 2, 4, 8}
tm := time.Now()
for _, n := range cases {
wg := sync.WaitGroup{}
wg.Add(n)
for i := 0; i < n; i++ {
go func() {
defer wg.Done()
for i := 0; i <= 1000000; i++ {
for j := 0; j < 5; j++ {
_ = chunk.NewColumn(types.NewFieldType(mysql.TypeLonglong), chunk.InitialCapacity)
}
}
}()
}
wg.Wait()
fmt.Println(time.Since(tm))
}
}
```
Test result:
=== RUN TestTTTT
561.074013ms
1.267392941s
2.360388094s
4.332767957s
--- PASS: TestTTTT (4.33s)

We can see the mutex for the first test, and find lock contentions.
![image](https://github.com/pingcap/tidb/assets/14054293/6c201ac4-81db-416b-9fc4-bb3f46f98bc0)

### 2. What did you expect to see? (Required)

### 3. What did you see instead (Required)

### 4. What is your TiDB version? (Required)
master, v6.5.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.