Support transform-based sort orders in ReplaceSortOrderAction (only identity is supported today)
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Is your feature request related to a problem or challenge?
`ReplaceSortOrderAction` (`transaction/sort_order.rs`) only supports sorting by a column's raw value. `asc`/`desc` hardcode `Transform::Identity` internally and only accept a column name — there is no way to declare a sort order using a transform (`bucket[N]`, `year`, `truncate[W]`, etc.).
Java's `SortOrderBuilder.asc/desc` accept a `Term`, which can be a plain column reference or a transform expression (`Expressions.bucket(name, n)`, `Expressions.year(name)`, ...), so this is a gap relative to the Java reference.
The underlying spec-level type is already ready for this: `SortField` has a general `transform: Transform` field, and `SortOrderBuilder::check_compatibility` already validates arbitrary transforms against the source column's type (including a `bucket[4]` round-trip in its own tests). The gap is only in the transaction-layer API surface.
### Describe the solution you'd like
Add `asc_with_transform` / `desc_with_transform` to `ReplaceSortOrderAction`, alongside the existing `asc`/`desc` (unchanged, still implying identity):
```rust
tx.replace_sort_order()
.asc_with_transform("event_time", Transform::Year, NullOrder::First)
.desc_with_transform("id", Transform::Bucket(16), NullOrder::Last);
```
Transform-compatibility with the source column's type should be checked at commit time (once the table schema is available), matching the timing of Java's `SortOrder.Builder.build()`.
### Scope note
This is a metadata-declaration gap only — which sort order (and transform) can be recorded on a table. Whether the write path actually sorts data to match a table's declared sort order is a separate, pre-existing gap unaffected by this: no writer in `crates/iceberg/src/writer/` reads `SortOrder` or sorts data today, for any sort order including the existing identity case.
### Willingness to contribute
I can contribute this independently.
Contributor guide
Research direction
Start in transaction/sort_order.rs with ReplaceSortOrderAction and its existing asc/desc methods, then read SortOrderBuilder::check_compatibility and the existing bucket[4] round-trip test. Add the transform-aware API while preserving identity behavior for the existing methods, and verify that transform compatibility is checked when the transaction commits against the table schema.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100