Performance Optimization: CPU and Memory Usage in Pipelined DML
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
There are some usages of CPU and memory allocations that could be optimized. Some of them may be intentional trade-offs and some are unintentional.
In the context of pipelined DML, the majority of latency comes from TiDB, making the execution speed in TiDB more critical to overall performance.
### Sub-tasks
- https://github.com/pingcap/tidb/issues/53973
- https://github.com/pingcap/tidb/issues/53942
- https://github.com/pingcap/tidb/issues/53837
- https://github.com/pingcap/tidb/issues/53763
- https://github.com/pingcap/tidb/issues/53714
- https://github.com/pingcap/tidb/issues/53660
- https://github.com/pingcap/tidb/issues/53604
- https://github.com/pingcap/tidb/pull/54308
- https://github.com/pingcap/tidb/issues/54604
- `GenKeyExistsErr`, in `insert on duplicate update key`
- Some `fmt.Sprintf`s, e.g. in tables.go
- `growSlice` in `genIndexValueVersion0`, in `insert on duplicate update key`
- `strings.Join` in `RequestSource.GetRequestSource`, in `insert on duplicate update key`
- `index.WithCtx`
- Allocating rows in `getRow`
- Lower the frequency of `onMemChange`
- batch more rows in `UpdateDeltaForTableFromColSlice`
- `SetMemoryFootprintChangeHook` in `Txn`
- `growSlice` in `Codec.decodeColumn`, in delete
- debug logs, even if they are not enabled
- `Error.Equal` and `IsErrNotFound`
Contributor guide
Assessment
This issue has not been assessed yet.