Optimize the use of pagestorage cache
@JinheLin is already working on this.
Since Aug 3, 2023.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Assume there is a hash join operator composed of two table scanning operators:
- table_build is used to scan the table and build a hash table for probing.
- table_probe is used to scan the table and probe the hash table built above.
Obviously, table_probe depends on table_build. These two table scanning operators can start concurrently in TiFlash, but table_probe will be blocked until table_build is finished.
In disaggregated mode, table scanning operators will occupy pagestorage cache space for each segment task when it starts and only release page cache space when corresponding segment task finishes.
If the total page cache space is small, table_probe could occupy all the pagestorage cache space and block to wait table_build to finish. However, in this case, table_build is blocked and wait for table_probe to release page cache space. Deadlock happened!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.