apache / apache/paimon-cpp

[Feature] Support Lance file format

已关闭
#313 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C++
星标
65
派生
25
平均合并
2 天 12 小时
30 天内合并 PR
80

描述

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon-cpp/issues) and found nothing similar.

### Motivation

Apache Paimon Java supports Lance as a data file format through the paimon-lance module. Paimon C++ currently cannot read or write tables configured with file.format=lance, which prevents Java and C++ engines from interoperating on those tables.

### Solution

Add optional Lance file format support to Paimon C++, aligned with the Java implementation and Lance 0.39.0:

- register a Lance file format factory behind a disabled-by-default CMake option;
- support reading and writing Lance files through the existing file format interfaces;
- support column projection, precise row selection, row counts, and file row-id reporting;
- validate the same supported and unsupported logical types as Java;
- add C++ round-trip tests and Java/C++ compatibility fixtures in both directions.

The implementation should follow the existing Mosaic Rust FFI integration and lifecycle patterns.

### Anything else?

Java reference module: https://github.com/apache/paimon/tree/master/paimon-lance

### Are you willing to submit a PR?

- [x] I am willing to submit a PR!

贡献指南

打开贡献指南

调研方向

首先阅读现有的 Mosaic Rust FFI 集成和生命周期模式,然后对比 paimon-lance 中的 Java 参考模块,并检查现有的文件格式接口。完成的标准是:一个默认禁用的可选 CMake 集成支持 Lance 的读取和写入、所需的投影和行选择、行元数据、逻辑类型验证、往返测试以及 Java/C++ 兼容性 fixtures。

由索引模型根据 Issue 内容生成。

评估

技术栈
cmake, cpp, java, rust
领域
data-engineering, databases
Issue 类型
功能
难度
5/5
预计耗时
一周以上
活跃度
活跃
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。