[Feature] [Core] Support concurrent lookup in LocalTableQuery
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
Currently, the lookup implementation in LocalTableQuery dose not support concurrent access and the operation to create Lookup File is very time-consuming. This will significantly affect the throughput of lookup in LocalTableQuery.
### Solution
Support async create LookupFile and introduce CompletableFuture here.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the LocalTableQuery lookup implementation and trace how Lookup Files are created. Review the current synchronous lookup path and determine where CompletableFuture-based creation must be integrated; done means concurrent lookup access is supported without the current throughput impact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100