[BP-53] Optimize write cache: supports configuring multiple caches
- Dominant language
- Java
- Stars
- 2k
- Forks
- 976
- Avg merge
- 6d 15h
- Merged PRs (30d)
- 7
Description
Discussion email: https://lists.apache.org/thread/25lz5g61vg3h4b3pzrny0vqhhyddy4rj
### Motivation
The current write cache has only 2 blocks, which has the following deficiencies:
1.Insufficient use of cache: When half of the write cache is flushed, it must wait for all the data in the write cache to be flushed before it can be released. With the same write cache size, if we divide the write cache into multiple blocks, the cache resources can be released faster, preventing the write request from waiting too long for the cache resources to be released;
2.The current write cache is allocated according to the configured data directory, so that the write cache resources between directories cannot be shared. When the write cache resources corresponding to one directory continue to be full, it is possible that the cache resources of other directories are still relatively low. Idle, so that the full utilization of the write cache is not used;
### Proposed Change
1.Two write caches become multi-cache blocks;
2.All data directories share write cache resources;
These changes are implemented in PR:https://github.com/apache/bookkeeper/pull/3260
The architecture diagram is as follows:
The effect comparison is as follows:

Contributor guide
Assessment
This issue has not been assessed yet.