apache / apache/iceberg-rust

Unrecognized transform names fail parsing instead of becoming Transform::Unknown

Open
#2,789 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

### Apache Iceberg Rust version

0.6.0 (latest version)

### Describe the bug

The V3 spec requires: "All v3 readers are required to read tables with unknown transforms, ignoring them."

`FromStr for Transform` returns an error for any unrecognized transform name — only the literal string `"unknown"` maps to `Transform::Unknown` — so loading a table whose partition spec or sort order uses a transform iceberg-rust does not know fails entirely. Names sharing a prefix with known transforms (e.g. `bucketv2[4]`) also fail. Additionally, `Transform::Unknown` does not retain the original name and displays as `"unknown"`, so rewriting metadata through the builder would corrupt the transform name.

For comparison, Java's `Transforms.fromString` falls back to `UnknownTransform`, which preserves the original string, and PyIceberg implemented the same behavior in apache/iceberg-python#3630.

### To Reproduce

```rust
use iceberg::spec::Transform;

let _ = "zorder".parse::(); // Err(DataInvalid), expected Transform::Unknown
let _ = "bucketv2[4]".parse::(); // Err(DataInvalid), expected Transform::Unknown
```

Loading table metadata containing such a transform fails the same way.

### Expected behavior

Unrecognized transform names parse as an unknown transform that preserves the original name, tables containing them load and scan (with no partition pruning on those fields), and the name round-trips through serialization. A fix likely means `Transform::Unknown(String)` (a breaking enum change) plus a parse fallback — noting the interaction with #2474, which is currently tightening the same parser to reject malformed parameters of known transforms.

### Willingness to contribute

I would be willing to contribute a fix for this bug with guidance from the Iceberg community

Contributor guide

Open the contributing guide

Research direction

Start with the FromStr implementation for Transform and trace how Transform::Unknown is represented, serialized, and used while loading partition specs or sort orders. Done means unrecognized names, including names sharing known prefixes, load and scan without pruning, preserve their original strings, and round-trip through metadata serialization while malformed known transforms remain governed by #2474.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.