apache / apache/fluss

[server] Support time-based log segment rolling (segment.ms)

Open
#3,833 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

Fluss currently rolls log segments only when size, index, or relative-offset limits are reached. Because retention and tiering skip the active segment, a low-throughput bucket can keep data in the active segment indefinitely.

### Solution

Add an optional `segment.ms`-like threshold to the existing append-time roll check. Use the replicated record-batch `commitTimestamp` so leader and follower choose the same segment boundary, and keep the option disabled by default.

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Locate the existing append-time log-segment roll check and the replicated record-batch commitTimestamp handling. Trace how leaders and followers choose segment boundaries, then verify that the optional time threshold is disabled by default and produces the same boundary on both sides.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.