apache / apache/iceberg-python

add_files support parquet files with field ids

Open
#3,556 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.1k
Forks
581
Avg merge
1d 17h
Merged PRs (30d)
78

Description

### Feature Request / Improvement

I am using rust to commit to iceberg rest catalogue, however that requires field id's to be written for commit, however when I try to commit those same files using pyIceberg ( In case of re-initialisation ).
This error is shown
```
fb0b1e42-2c49-434c-b39a-da4586e7ef05-die_info.parquet because it has field IDs. `add_files` only supports addition of files without field_ids
```
If it is not assigned to someone I believe I can work on this.
Thanks!s

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the add_files entry point and reading how it currently handles Parquet files with field IDs. The work is done when add_files accepts the field-ID-bearing files described in the report while preserving existing support for files without field IDs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
54/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.