[Java][C] sliced RecordBatch offset info is lost when imported from c-data
- 主要語言
- Java
- 星號
- 94
- 分支
- 152
- 平均合併
- 3 天 16 小時
- 30 天內合併 PR
- 11
描述
### Describe the bug, including details regarding any error messages, version, and platform.
Reproduced on latest arrow release (16.0)
When importing a sliced RecordBatch from c to java
On c side:
```
auto sliced_record_batch = original_record_batch->Slice(/*offset=*/8, /*length=*/2);
arrow::ExportRecordBatch(sliced_record_batch, arrow_array_ptr);
```
On java side:
```
ArrowArray arrowArray = ArrowArray.allocateNew(allocator);
Data.importIntoVectorSchemaRoot(allocator, arrowArray, vectorSchemaRoot, null);
```
The imported vectorSchemaRoot maintains the correct length(which is 2), but the offset info (which is 8) is not respected, hence the content of the imported vectorSchemaRoot points to the first 2 rows of the original_record_batch, while the desired content is sliced_record_batch.
I'm not familiar with arrow code, but it seems that the offset info is actually present in org.apache.arrow.c.ArrowArray.Snapshot, but org.apache.arrow.c.ArrayImporter ignores the offset in org.apache.arrow.c.ArrayImporter.doImport(ArrowArray.Snapshot)
### Component(s)
Java
貢獻指南
研究方向
首先閱讀 org.apache.arrow.c.ArrayImporter.doImport(ArrowArray.Snapshot) 以及 org.apache.arrow.c.ArrowArray.Snapshot 中的 offset 資訊。使用 C 端切片後的 RecordBatch 重現 Java 匯入,然後驗證匯入的 vectorSchemaRoot 包含從 slice offset 開始的資料列,而不是原始 batch 的前幾列。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- c, java
- 領域
- data-engineering
- Issue 類型
- 缺陷
- 難度
- 3/5
- 預估耗時
- 1-2 天
- 活躍度
- 停滯
- 描述清晰度
- 描述清楚
- 新手友好度
- 45/100