pingcap / pingcap/tidb

Lower the default value of tidb_max_chunk_size from 1024 to 32

Open
#50,414 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement
we've seen cases where we need to lower the value of `tidb_max_chunk_size` for OLTP system, by lower the value of `tidb_max_chunk_size`, the performance and stability of the system can be improved, here are some lessons learned.
1. lower the memory allocation pressure
2. When the limit pushdown to TiKV is not feasible due to various limitations, lower the value of `tidb_max_chunk_size`, TiKV can sent back fewer rows back to TiDB, which improve the performance. e.g. https://github.com/pingcap/tidb/issues/46755

So we recommend to lower the value of `tidb_max_chunk_size` from 1024 to 32

We also bench `tidb_max_chunk_size` with 32 vs 1024, the performance degradation for OLAP workload on TiKV is between 8% and 34%.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.