apache / apache/gluten

[VL][FLINK] Velox4J query plan binary serialization

Open
#9,567 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

Gluten-flink performs Java / Kyro serializations on the Velox4J query plan. This currently requires an [API change](https://github.com/bigo-sg/velox4j/blob/1239b1ea42961cacbe38294a01304f2acc0f10f3/src/main/java/io/github/zhztheplayer/velox4j/serde/NativeBean.java#L5-L6) in `bigo-sg/velox4j` to make `NativeBean` extend `Serializable`.

This change is reasonable but so far there are some bean definitions in Velox4J not compatible with Java `Serializable`. The beans typically have Java optionals which are not serializable by design as class members. We should find a better way to address the issue and make the beans Java-serializable in Velox4J.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.