apache / apache/iceberg-python

Implement iterator to lazily go through the paged response

Open
#3,365 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.1k
Forks
581
Avg merge
1d 17h
Merged PRs (30d)
77

Description

### Feature Request / Improvement

Recently the support for paged response was added to the list-view operationr: https://github.com/apache/iceberg-python/pull/3349. Right now it will just iterate over all the pages, instead it would be good to see if we can call the next page lazily to avoid many network calls: https://github.com/apache/iceberg-python/pull/3349#issuecomment-4462173769

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the paged list-view implementation introduced in PR 3349 and its linked discussion. Trace how pages are currently iterated, then verify that consuming the iterator requests each next page only when needed instead of fetching all pages upfront.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.