apache / apache/iceberg-python
Async API for RestCatalog
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 581
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 78
Description
### Feature Request / Improvement
RestCatalog currently uses synchronous HTTP calls, while most catalog operations are network io-bound. Native async support would help async consumers and high-concurrency use cases in general.
We could introduce a separate AsyncRestCatalog, preserving the existing synchronous API.
asyncio.to_thread()/running in a dedicated executor is a possible workaround but doesn't give the same benefits as native support as we'd still have the limitations of the sync client (one worker thread per in-flight request being the most annoying in this case).
We can start small without necessarily the full ops coverage, wdyt?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the existing RestCatalog implementation and map which catalog operations currently use synchronous HTTP calls. Define the initial AsyncRestCatalog scope while preserving the synchronous API; the issue leaves the exact operation coverage and validation approach open.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100