apache / apache/fluss

[lake/iceberg] Support multi-bucket-key tables

Open
#4,069 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

Fluss supports composite bucket keys (e.g., `DISTRIBUTED BY (tenant_id, user_id)`). Paimon handles these natively. Iceberg's integration currently throws `UnsupportedOperationException` for multi-key tables (restriction added in #2499).

Since Fluss already computes the composite hash and stores the result in the `__bucket` system column, the Iceberg partition spec can use `identity(__bucket)` for multi-key cases without requiring Iceberg to understand the composite hash.

### Solution

Remove the `bucketKeys.size() > 1` check in `IcebergLakeCatalog.createPartitionSpec()`. For multi-key tables, use `identity(BUCKET_COLUMN_NAME)` partitioning.

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in IcebergLakeCatalog.createPartitionSpec() and inspect the existing bucket-key restriction and partition-spec handling. Verify the multi-key path uses identity(__bucket), then run the relevant Iceberg integration tests or add coverage for a table with composite bucket keys; done means multi-key tables no longer throw UnsupportedOperationException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.