[Improvement] leverage concurrent iteration of fileScanTasks to improve table scan plan performance in Iceberg REST server
- Dominant language
- Java
- Stars
- 3.2k
- Forks
- 935
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 298
Description
### What would you like to be improved?
for (FileScanTask fileScanTask : fileScanTasks) —— would using a parallel stream here be better? During actual usage we found this block to be unstable in terms of latency, and our trace revealed the costs is below
+---[min=0.002104ms,max=3137.10955ms,total=3159.839086ms,count=476] java.util.Iterator:hasNext()
#9047
### How should we improve?
_No response_
Contributor guide
Research direction
Start by locating the for-loop over FileScanTasks in the Iceberg REST server's table scan plan path and review issue #9047 for context. Compare sequential and concurrent iteration using the reported latency trace or a suitable benchmark; done means improved and more stable scan-plan latency without changing results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100