apache / apache/fluss

Support configurable time partition format for auto-partitioned tables

Open
#3,191 1 comment 0 reactions 1 assignee Claimed by @wattt3 View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

## Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

## Motivation
Fluss currently fixes the partition value format based on `table.auto-partition.time-unit`. For example, `DAY` uses `yyyyMMdd`, `HOUR` uses `yyyyMMddHH`, and users cannot configure another format.

This becomes awkward when Fluss is used together with Paimon. Paimon commonly uses `yyyy-MM-dd` style date partitions by default, while Fluss expects `yyyyMMdd`. As a result, users need extra conversion logic or have to accept inconsistent partition naming between Fluss and Paimon.

Making the time partition format configurable would improve interoperability and reduce friction for lake integration scenarios, especially when users want Fluss and Paimon to use the same partition naming convention.

## Solution
Add a table option for configuring the time partition formatter, for example `table.auto-partition.time-format` or similar.

Expected behavior:
- Keep the current hard-coded formats as the default to preserve backward compatibility.
- Allow users to override the formatter when using time-based auto partitioning, for example `yyyy-MM-dd` for `DAY` partitions.
- Use the configured formatter consistently for partition generation, validation, retention, and pre-creation logic.
- Document the supported formatter patterns and any compatibility constraints with different time units.

## Anything else?
There is already related formatter support on the Paimon side, such as `paimon.partition.timestamp-formatter`, so exposing a similar capability in Fluss would make Fluss-Paimon integration more natural.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.