ClickHouse / ClickHouse/ClickHouse
What about write-back cache for remoteFS
Open
comp-filesystem-cache
external
feature
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
Hello @kssenii, we have a write-through cache for remoteFS in ClickHouse. What about a write-back cache?
We can track dirty file segments for a moment(eg. 5 seconds), and keeping flush to remote storage in the background. Just like pd-flush thread in kernel filesystem flush dirty page-cache.
We can get benefit from keeping file segments in cache for a moment when user insert small batch frequently, then the small batch data will be merged and delete in cache, not upload to remoteFS.
Contributor guide
Assessment
This issue has not been assessed yet.