[spark] Umbrella issue for scan optimization
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Motivation
This is is an umbrella issue to track the basic scan optimization for spark connector.
- [x] #3117
- [ ] pk table predicate pushdown
- [x] #3239
- [x] #3241
- [ ] column pruning
- [x] #3215
- [ ] batch read instead of row read
### Solution
_No response_
### Anything else?
_No response_
### Willingness to contribute
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the linked issues #3117, #3239, #3241, and #3215 to understand the Spark connector scan optimization work already tracked. The remaining scope is primary-key table predicate pushdown, column pruning, and batch reads instead of row reads; done means the applicable unchecked items are completed and verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100