apache / apache/bookkeeper

[BP-53] Optimize write cache: supports configuring multiple caches

Open
#3,322 0 comments 0 reactions 0 assignees View on GitHub
type/proposal
Dominant language
Java
Stars
2k
Forks
976
Avg merge
6d 15h
Merged PRs (30d)
7

Description

Discussion email: https://lists.apache.org/thread/25lz5g61vg3h4b3pzrny0vqhhyddy4rj

### Motivation
The current write cache has only 2 blocks, which has the following deficiencies:
1.Insufficient use of cache: When half of the write cache is flushed, it must wait for all the data in the write cache to be flushed before it can be released. With the same write cache size, if we divide the write cache into multiple blocks, the cache resources can be released faster, preventing the write request from waiting too long for the cache resources to be released;

2.The current write cache is allocated according to the configured data directory, so that the write cache resources between directories cannot be shared. When the write cache resources corresponding to one directory continue to be full, it is possible that the cache resources of other directories are still relatively low. Idle, so that the full utilization of the write cache is not used;

### Proposed Change
1.Two write caches become multi-cache blocks;
2.All data directories share write cache resources;
These changes are implemented in PR:https://github.com/apache/bookkeeper/pull/3260

The architecture diagram is as follows:
image
The effect comparison is as follows:
image
image

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.