feat(core): support multiple ordering fields end-to-end
- Dominant language
- Rust
- Stars
- 279
- Forks
- 67
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 40
Description
## Description
`hoodie.table.ordering.fields` supports comma-separated multiple fields (e.g., `ts,seq`) for lexicographic record ordering. The config parsing already handles this as a `List`, but the value is currently rejected at validation time (`UnsupportedValue`) because downstream code only handles a single ordering field.
## What needs to change
To support multiple ordering fields, the following code paths need updates:
### Data record merging (`merge/record_merger.rs`)
- `lexsort_to_indices` should build sort columns from all ordering fields instead of a single one
- Event time converter and extraction should handle multiple columns
### Ordering comparison (`merge/ordering.rs`)
- `process_batch_for_max_orderings` should extract and compare across all ordering fields
- `MaxOrderingInfo` event time comparison should be multi-column
### Record extraction (`record/mod.rs`)
- `create_event_time_ordering_converter` and `extract_event_time_ordering_values` should accept `&[String]` instead of `&str`
### Delete record handling
- Hudi delete records (`HoodieDeleteRecord.avsc`) only store a single `orderingVal`. Need to decide how multi-field ordering interacts with delete records — likely compare on the first field only for data-vs-delete comparisons, since the delete format is a Hudi spec constraint.
### Config validation (`config/table.rs`)
- Remove the `UnsupportedValue` guard for multiple ordering fields in `parse_value`
## Context
Introduced in #598 — config alias support and `OrderingFields` rename.
Contributor guide
Research direction
Start with config/table.rs to trace the current UnsupportedValue validation, then read merge/record_merger.rs, merge/ordering.rs, and record/mod.rs at the named functions. Review HoodieDeleteRecord.avsc before deciding how delete records participate in multi-field comparisons. Done means multiple ordering fields pass validation and are handled through merging, ordering comparison, extraction, and delete-record behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100