File Type Support
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Proposed Change
Authors: Talat Uyarer, Alex Stephen, Sung Yun, Gaurav Soni
Add file type to Iceberg v4
Add a new file primitive type to the Iceberg v4 spec for referencing a range of bytes, stored either inline or in an external object.
The type and its value semantics are defined by the Parquet FILE logical type (apache/parquet-format#585); The Iceberg spec covers the schema representation, field-ID reservation, statistics behavior, and file-format mappings (Parquet/Avro/ORC).
Spec PR #17918
Reference Impl PR #17821
### Proposal document
https://docs.google.com/document/d/1BdA4G-38GoVcpX6UY4UI-QA1MvrDnH6uSKeCJBadILQ/edit?tab=t.pnp3scifr79p#heading=h.v4a92wm7mz0o
### Specifications
- [x] Table
- [ ] View
- [ ] REST
- [ ] Puffin
- [ ] Encryption
- [ ] Other
Contributor guide
Research direction
Start with the proposal document, then compare Iceberg v4 spec PR #17918 and reference implementation PR #17821. Review the checked Table item and the unchecked View, REST, Puffin, and Encryption items; done means the file primitive is consistently covered in the applicable specifications and implementation, including schema, field-ID, statistics, and format mappings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100