apache / apache/arrow-java

[Java] Integrating `variadicBufferCounts` to `RecordBatch` in Java

未關閉
#86 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Java
星號
94
分支
152
平均合併
3 天 16 小時
30 天內合併 PR
11

描述

### Describe the enhancement requested

Once the `Utf8View` and `BinaryView` have been introduced, we have added a new attribute to the `RecordBatch` [spec](https://github.com/apache/arrow/blob/main/format/Message.fbs#L117). Once this change is integrated, we need to update the existing interfaces which uses this and related modules which use those interfaces. To introduce this change, it would be better to include this by introducing them component wise so that it can be tested in a streamlined manner.

Following are list of modules and their classes which are related with the changes done to update the `RecordBatch` specification.

### Vector

- [ ] `JsonFileReader`
- [ ] `JsonFileWriter`
- [ ] `ArrowReader`
- [X] `TestValueVector`
- [X] `TestVarcharViewVector`
- [ ] `TestVectorUnloadLoad`
- [ ] `TestArrowReaderWriter`
- [ ] `TestRoundtrip`
- [ ] `TestJsonFile`
- [ ] `ArrowRecordBatchBenchmarks`
- [ ] `MessageSerializerTest`
- [ ] `ArrowDictionaryBatch`
- [ ] `ArrowStreamReader`
- [ ] `TestValidateVector`

### Performance

- [ ] `VectorLoaderBenchmark`
- [ ] `VectorUnloaderBenchmarks`

### C

- [ ] `RoundtripTest`
- [ ] `StreamTest`

### Dataset

- [ ] `TestDataset`
- [ ] `TestFileSystemDataset`
- [ ] `ArrowScannerReader`
- [ ] `NativeScanner`

### Flight

- [ ] `TestFlightClient`
- [ ] `TestDoExchange`
- [ ] `IntegrationTestClient`
- [ ] `FlightStream`
- [ ] `IntegrationProducer`
- [ ] `ExpirationTimeDoGetScenario`
- [ ] `VectorSchemaRootTransformerTest`
- [ ] `FlightSQLExample`
- [ ] `TestBasicOperation`

### Compression

- [ ] `TestCompressionCodec`

### Gandiva

- [ ] `ProjectorTest`
- [ ] `ProjectorDecimalTest`
- [ ] `FilterTest`
- [ ] `FilterProjectTest`
- [ ] `BaseEvaluatorTest`

### Component(s)

Java

貢獻指南

開啟貢獻指南

研究方向

從 RecordBatch 規範以及受 variadicBufferCounts 影響的現有 Java 介面開始,接著逐個元件處理 unchecked Vector、Performance、C、Dataset、Flight、Compression 和 Gandiva 類別。將已完成的 checked TestValueVector 和 TestVarcharViewVector 工作作為脈絡。列出的模組都已一致地更新,且其相關測試通過,即表示完成。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
java
領域
data
Issue 類型
功能
難度
5/5
預估耗時
一週以上
活躍度
停滯
描述清晰度
需要釐清
新手友好度
20/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。