discussion: align v1/v2 metadata parsing with Java/PyIceberg on v3-only fields
- Dominant language
- Go
- Stars
- 463
- Forks
- 232
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 121
Description
Follow-up from #1069, open question.
#1069 hard-errors on `next-row-id` / `encryption-keys` in v1/v2 metadata. None of the reference clients do this — Java reads `next-row-id` only when `formatVersion >= 3` and defaults otherwise, has no version guard on `encryption-keys` at all; PyIceberg and iceberg-rust both drop unknown fields. So we're the only one that rejects here.
Doesn't bite today because v3 isn't widely deployed. Once downgrade tooling shows up, a v2 file with stray v3 fields reads everywhere except Go.
Strict surface has value (mis-versioned metadata is by definition wrong, and it catches our own writer if it ever leaks v3 fields), but forward-compat is also a spec goal. If we keep it strict, a code comment near `rejectV3OnlyFields` capturing the intentional divergence would help future readers not read it as a parity bug.
Contributor guide
Research direction
Start with #1069 and the metadata parsing path around `rejectV3OnlyFields`; compare the current behavior with the Java, PyIceberg, and iceberg-rust behavior described here. Done means resolving the strictness versus forward-compatibility question and recording the chosen behavior, including a comment near the guard if strict rejection remains.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100