pingcap / pingcap/tiflash

Optimize append write

Open
#1,019 0 comments 0 reactions 1 assignee View on GitHub

@flowbehappy is already working on this.

Since Aug 18, 2020.

Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

If tables do not have a primary key, then TiDB will generate a hidden primary key, i.e. handle. And handle is auto increasing during the insertion. This causes the data to only append to the end of the latest segment. Even though we can use SHARD_ROW_ID_BITS to scatter data, sometimes it is still causing pressure to write. The main issue is that the delta merge of segments may not be able to do fast enough, and cause write stall.

Currently, delta merge needs to combine the data of delta and stable. But since most of the data in stable has not been updated, we can skip those data to speed up delta merge. To do this we need stable to support multiple DTFiles.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.