apache / apache/iceberg-python

feat: refactor rest signer in fileio

Open
#2,862 4 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.1k
Forks
581
Avg merge
1d 17h
Merged PRs (30d)
78

Description

### Feature Request / Improvement

Context: https://github.com/apache/iceberg-python/pull/2846#discussion_r2645920570

Currently S3 signer is tightly coupled with FileIO. This is awkward since the S3 Signer requires interaction with remote service (i.e. REST Catalog) and reuse its auth manager.

We can refactor here and create the signer in REST Catalog and pass to FileIO by overriding its `_load_file_io` function.

Also see [_init_sigv4](https://github.com/apache/iceberg-python/blob/a99dcada431a06329959cb7f71bf961343256bb0/pyiceberg/catalog/rest/__init__.py#L409-L459) for inspiration

Contributor guide

No contributing guide indexed for this repository

Research direction

Read the discussion in PR 2846 and the `_init_sigv4` implementation in `pyiceberg/catalog/rest/__init__.py`. Trace REST Catalog's `_load_file_io` path and how the S3 signer currently couples to FileIO. Done means the REST Catalog creates the signer using its auth manager and passes it to FileIO through the override.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, authentication, backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.