apache / apache/hudi-rs

Feature Request: Support Clustering in hudi-rs

Open
#400 0 comments 1 reaction 0 assignees View on GitHub
feature p1
Dominant language
Rust
Stars
279
Forks
67
Avg merge
3d 1h
Merged PRs (30d)
40

Description

### Feature Description

Clustering is a core optimization feature in Apache Hudi, widely used to manage small files and improve query performance.

I’d love to see support for clustering in hudi-rs, which could handle this efficiently thanks to Rust’s performance. This would enable production-grade optimization workflows in native Rust pipelines.

### Why this matters
- Performance: Rust is expected to make compute-intensive operations like clustering much more performant
- Ease of migration: Users who are using standalone clustering should be able to migrate to hudi-rs clustering easily
- Ecosystem trend: Similar efforts are emerging in other formats, e.g. [Iceberg compaction](https://github.com/nimtable/iceberg-compaction/tree/main).

### Suggested Scope
Initial support might include:
- Reading clustering plans
- Executing clustering as a standalone action
- Supporting inline clustering in write paths (optional follow-up)

### Prerequisites
- hudi-rs doesn't have write support overall, it needs to be able to write and commit data before we implementing complicated table services

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing write and commit support in hudi-rs, then trace how table services could read clustering plans and execute a standalone clustering action. The issue identifies inline clustering as an optional follow-up; done would require write and commit support plus an agreed initial clustering scope, but no files or tests are named.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.