apache / apache/iceberg-rust

Support transform-based sort orders in ReplaceSortOrderAction (only identity is supported today)

Open
#2,764 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### Is your feature request related to a problem or challenge?

`ReplaceSortOrderAction` (`transaction/sort_order.rs`) only supports sorting by a column's raw value. `asc`/`desc` hardcode `Transform::Identity` internally and only accept a column name — there is no way to declare a sort order using a transform (`bucket[N]`, `year`, `truncate[W]`, etc.).

Java's `SortOrderBuilder.asc/desc` accept a `Term`, which can be a plain column reference or a transform expression (`Expressions.bucket(name, n)`, `Expressions.year(name)`, ...), so this is a gap relative to the Java reference.

The underlying spec-level type is already ready for this: `SortField` has a general `transform: Transform` field, and `SortOrderBuilder::check_compatibility` already validates arbitrary transforms against the source column's type (including a `bucket[4]` round-trip in its own tests). The gap is only in the transaction-layer API surface.

### Describe the solution you'd like

Add `asc_with_transform` / `desc_with_transform` to `ReplaceSortOrderAction`, alongside the existing `asc`/`desc` (unchanged, still implying identity):

```rust
tx.replace_sort_order()
.asc_with_transform("event_time", Transform::Year, NullOrder::First)
.desc_with_transform("id", Transform::Bucket(16), NullOrder::Last);
```

Transform-compatibility with the source column's type should be checked at commit time (once the table schema is available), matching the timing of Java's `SortOrder.Builder.build()`.

### Scope note

This is a metadata-declaration gap only — which sort order (and transform) can be recorded on a table. Whether the write path actually sorts data to match a table's declared sort order is a separate, pre-existing gap unaffected by this: no writer in `crates/iceberg/src/writer/` reads `SortOrder` or sorts data today, for any sort order including the existing identity case.

### Willingness to contribute

I can contribute this independently.

Contributor guide

Open the contributing guide

Research direction

Start in transaction/sort_order.rs with ReplaceSortOrderAction and its existing asc/desc methods, then read SortOrderBuilder::check_compatibility and the existing bucket[4] round-trip test. Add the transform-aware API while preserving identity behavior for the existing methods, and verify that transform compatibility is checked when the transaction commits against the table schema.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.