[Java] VectorSchemaRoot convertion into ArrowArrayStream
- 主要語言
- Java
- 星號
- 94
- 分支
- 152
- 平均合併
- 3 天 16 小時
- 30 天內合併 PR
- 11
描述
### Describe the enhancement requested
DuckDB support ArrowArrayStream as virtual table. JDBC driver of DuckDB has "register" ArrowStream method on DuckDBConnection.
VectorSchemaRoot convertion into ArrowArrayStream object will allows perform complex SQL transformation with CTE,window function on existing Arrow data from java process without data format transformation and also as workaround to write parquet file from VectorSchemaRoot.
```
try (DuckDBConnection connection = (DuckDBConnection) DriverManager.getConnection("jdbc:duckdb:")){
connection.registerArrowStream("arrow_stream", arrowArrayStream);
try (Statement preparedStatement = connection.createStatement()){
preparedStatement.executeUpdate("COPY (SELECT * FROM arrow_stream) TO '"+filePath+"' (FORMAT 'PARQUET', CODEC 'ZSTD')");
}
}
```
@lidavidm do you have idea how to implement such conversion?
### Component(s)
Java
貢獻指南
研究方向
從 Java 元件和 issue 中的 DuckDB JDBC 範例開始,然後結合 DuckDBConnection.registerArrowStream 檢查 VectorSchemaRoot 和 ArrowArrayStream API。完成的標準是:VectorSchemaRoot 可以轉換為 ArrowArrayStream,且所示的註冊和 SQL 工作流程可以在不進行格式轉換的情況下消費它。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- java
- 領域
- data
- Issue 類型
- 功能
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100