ClickHouse / ClickHouse/ClickHouse

What about write-back cache for remoteFS

Open
#37,931 2 comments 0 reactions 0 assignees View on GitHub
comp-filesystem-cache external feature
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

Hello @kssenii, we have a write-through cache for remoteFS in ClickHouse. What about a write-back cache?

We can track dirty file segments for a moment(eg. 5 seconds), and keeping flush to remote storage in the background. Just like pd-flush thread in kernel filesystem flush dirty page-cache.

We can get benefit from keeping file segments in cache for a moment when user insert small batch frequently, then the small batch data will be merged and delete in cache, not upload to remoteFS.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.