apache / apache/fluss

Reduce CoordinatorServer memory footprint for kv snapshot

Open
#528 0 comments 0 reactions 0 assignees View on GitHub
component=kv
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.

### Description

Currently, CoordinatorServer will maintain the kv snapshots in memory to delete unused kv snapshot file via reference count which require to maintail all kv files(represent by `KvFileHandle`) in memory which is memory cost. We can reduce memory footprint by following ways:

### Willingness to contribute

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating CoordinatorServer and KvFileHandle and tracing how kv snapshots are retained and deleted through reference counting. Review the existing snapshot cleanup path; done means unused kv snapshot files are still deleted correctly while CoordinatorServer holds less snapshot state in memory.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.