apache / apache/fluss

[lake/lance] Support Union Read query against Lance table

Open
#2,715 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

Batch / Union Read query against Lance table is [currently unimplemented](https://github.com/apache/fluss/blob/575db63687793be0086c63f6793bce683e1281f0/fluss-lake/fluss-lake-lance/src/main/java/org/apache/fluss/lake/lance/LanceLakeStorage.java#L49)

```
public class LanceLakeStorage implements LakeStorage {
...
@Override
public LakeSource createLakeSource(TablePath tablePath) {
throw new UnsupportedOperationException("Not implemented");
}
}
```

Implementing LakeSource here will allow batch / union read from Flink SQL.

### Solution

_No response_

### Anything else?

_No response_

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in fluss-lake/fluss-lake-lance/src/main/java/org/apache/fluss/lake/lance/LanceLakeStorage.java at createLakeSource, then inspect the LakeSource interface and related lake storage implementations. Determine the requirements for batch and union reads from Flink SQL, and consider the work complete when LanceLakeStorage provides a working LakeSource for those queries.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.