[RestCatalog] Support Page Sizes
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Is your feature request related to a problem or challenge?
By [the REST spec](https://iceberg.apache.org/docs/latest/catalog-properties/#rest-catalog-properties), a REST catalog may return the `rest-page-size` property in a `/v1/config` response. The catalog client is then supposed to use it and provide it along with list requests via the `?pageSize` query parameter.
Today, the Rust `RestCatalog` client doesn't look at the `rest-page-size` property and never sends a `pageSize` query parameter. It therefore depends on the catalog server to implement an implicit default page size.
### Describe the solution you'd like
The spec already prescribes this fairly precisely.
We'd use a precedence order similar to other properties:
1. catalog default from `/v1/config`: `{"defaults": {"rest-page-size": "1000"}}` (if provided)
2. overridden by client side config: `with_page_size(1000)` (else `None` or some client-side default[^1])
3. overridden by catalog overrides from `v1/config`: `{"overrides": {"rest-page-size": "1000"}}` (if provided)
[^1]: IIUC only `iceberg-go` currently sets a client-side default (20). Java and Python don't.
### Willingness to contribute
I can contribute to this feature independently
Contributor guide
Research direction
Start with the Rust RestCatalog handling for /v1/config and list requests. Trace how catalog defaults, with_page_size(1000), and catalog overrides should be combined, then verify that list requests send the resulting value as the pageSize query parameter; done means the REST spec precedence is respected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100