apache / apache/iceberg-python
Add option to pass storage config to polars lazy read
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 581
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 78
Description
### Feature Request / Improvement
I was not able to use the `table.to_polars()` method because there is no option to include a storage_options param which is mentioned in the [polars docs](https://docs.pola.rs/api/python/dev/reference/api/polars.scan_iceberg.html).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the implementation of table.to_polars() and trace how it invokes Polars' lazy read path. Check the Polars scan_iceberg documentation linked in the issue for the storage_options interface. Done means callers can provide storage_options through table.to_polars() and the option reaches the lazy read operation, with relevant tests updated or added.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100