pingcap / pingcap/tiflash

Optimize the use of pagestorage cache

Open
#7,883 0 comments 0 reactions 1 assignee View on GitHub

@JinheLin is already working on this.

Since Aug 3, 2023.

type/enhancement
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

Assume there is a hash join operator composed of two table scanning operators:

  • table_build is used to scan the table and build a hash table for probing.
  • table_probe is used to scan the table and probe the hash table built above.

Obviously, table_probe depends on table_build. These two table scanning operators can start concurrently in TiFlash, but table_probe will be blocked until table_build is finished.

In disaggregated mode, table scanning operators will occupy pagestorage cache space for each segment task when it starts and only release page cache space when corresponding segment task finishes.

If the total page cache space is small, table_probe could occupy all the pagestorage cache space and block to wait table_build to finish. However, in this case, table_build is blocked and wait for table_probe to release page cache space. Deadlock happened!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.