Process manifests parallelly using runtime API when deleting data
Open
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
This is executing concurrently, not parallelly. What we need is runtime api.
_Originally posted by @blackmwk in https://github.com/apache/iceberg-rust/pull/2232#discussion_r2978543400_
Contributor guide
Research direction
Start with the discussion linked from PR #2232 and trace the existing manifest-deletion path. Identify how it currently executes concurrently and where the runtime API should be involved; done means manifest processing uses the runtime API for parallel execution rather than only concurrency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100