apache / apache/fluss

[kv] Implement KvTablet RocksDB Lazy Open

Open
#2,913 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

When a tablet server hosts thousands of KV buckets, all RocksDB instances are opened eagerly at startup, consuming significant memory. In practice, many buckets may receive no traffic for hours or even days — especially in multi-tenant or high-partition-count deployments. These idle RocksDB instances waste memory without serving any requests.

### Solution

Defer RocksDB open to first access. Each KV tablet starts as a lightweight sentinel (no RocksDB). When the first read/write arrives, RocksDB is opened on demand. After a configurable idle timeout, the tablet is automatically released back to the lightweight state, preserving local SST files for fast reopen.

### Anything else?

_No response_

### Willingness to contribute

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating KvTablet and tracing how its RocksDB instances are opened and released during tablet startup and request handling. Compare the current eager lifecycle with the requested first-access open and idle release behavior; done means idle tablets remain lightweight, reopen on the first read or write, and preserve local SST files.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.