Optimize append write
@flowbehappy is already working on this.
Since Aug 18, 2020.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
If tables do not have a primary key, then TiDB will generate a hidden primary key, i.e. handle. And handle is auto increasing during the insertion. This causes the data to only append to the end of the latest segment. Even though we can use SHARD_ROW_ID_BITS to scatter data, sometimes it is still causing pressure to write. The main issue is that the delta merge of segments may not be able to do fast enough, and cause write stall.
Currently, delta merge needs to combine the data of delta and stable. But since most of the data in stable has not been updated, we can skip those data to speed up delta merge. To do this we need stable to support multiple DTFiles.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.