apache / apache/iceberg-python

Async API for RestCatalog

Open
#3,861 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.1k
Forks
581
Avg merge
1d 17h
Merged PRs (30d)
78

Description

### Feature Request / Improvement

RestCatalog currently uses synchronous HTTP calls, while most catalog operations are network io-bound. Native async support would help async consumers and high-concurrency use cases in general.

We could introduce a separate AsyncRestCatalog, preserving the existing synchronous API.

asyncio.to_thread()/running in a dedicated executor is a possible workaround but doesn't give the same benefits as native support as we'd still have the limitations of the sync client (one worker thread per in-flight request being the most annoying in this case).

We can start small without necessarily the full ops coverage, wdyt?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the existing RestCatalog implementation and map which catalog operations currently use synchronous HTTP calls. Define the initial AsyncRestCatalog scope while preserving the synchronous API; the issue leaves the exact operation coverage and validation approach open.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.