pingcap / pingcap/tidb

Performance Optimization: CPU and Memory Usage in Pipelined DML

Open
#54,444 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement

There are some usages of CPU and memory allocations that could be optimized. Some of them may be intentional trade-offs and some are unintentional.

In the context of pipelined DML, the majority of latency comes from TiDB, making the execution speed in TiDB more critical to overall performance.

### Sub-tasks

- https://github.com/pingcap/tidb/issues/53973
- https://github.com/pingcap/tidb/issues/53942
- https://github.com/pingcap/tidb/issues/53837
- https://github.com/pingcap/tidb/issues/53763
- https://github.com/pingcap/tidb/issues/53714
- https://github.com/pingcap/tidb/issues/53660
- https://github.com/pingcap/tidb/issues/53604
- https://github.com/pingcap/tidb/pull/54308
- https://github.com/pingcap/tidb/issues/54604
- `GenKeyExistsErr`, in `insert on duplicate update key`
- Some `fmt.Sprintf`s, e.g. in tables.go
- `growSlice` in `genIndexValueVersion0`, in `insert on duplicate update key`
- `strings.Join` in `RequestSource.GetRequestSource`, in `insert on duplicate update key`
- `index.WithCtx`
- Allocating rows in `getRow`
- Lower the frequency of `onMemChange`
- batch more rows in `UpdateDeltaForTableFromColSlice`
- `SetMemoryFootprintChangeHook` in `Txn`
- `growSlice` in `Codec.decodeColumn`, in delete
- debug logs, even if they are not enabled
- `Error.Equal` and `IsErrNotFound`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.