apache / apache/fluss

[spark] Umbrella issue for scan optimization

Open
#3,213 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

This is is an umbrella issue to track the basic scan optimization for spark connector.

- [x] #3117
- [ ] pk table predicate pushdown
- [x] #3239
- [x] #3241
- [ ] column pruning
- [x] #3215
- [ ] batch read instead of row read

### Solution

_No response_

### Anything else?

_No response_

### Willingness to contribute

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the linked issues #3117, #3239, #3241, and #3215 to understand the Spark connector scan optimization work already tracked. The remaining scope is primary-key table predicate pushdown, column pruning, and batch reads instead of row reads; done means the applicable unchecked items are completed and verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.