apache / apache/arrow-java

[C++][Java] When i use DatasetFileWriter::write method write a file, I can't specify a file name

未關閉
#64 4 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
Type: usage
主要語言
Java
星號
94
分支
152
平均合併
3 天 16 小時
30 天內合併 PR
11

描述

### Describe the usage question you have. Please include as many useful details as possible.

I use this static method to write a parquet file into fs, I give empty partition array, so it will be in a file; and i give the `baseNameTemplate` arg is "test.parquet" to specify the filename, but it leads to an error: `basename_template did not contain '{i}'`.
```java
public static void write(BufferAllocator allocator, ArrowReader reader, FileFormat format, String uri,
String[] partitionColumns, int maxPartitions, String baseNameTemplate) {
try (final ArrowArrayStream stream = ArrowArrayStream.allocateNew(allocator)) {
Data.exportArrayStream(allocator, reader, stream);
JniWrapper.get().writeFromScannerToFile(stream.memoryAddress(),
format.id(), uri, partitionColumns, maxPartitions, baseNameTemplate);
}
}
```
read the cpp jni code and cpp dataset's code, the FileSystemDatasetWriteOptions::basename_template seems not supported to specify the file name without `{i}`.
```cpp
/// \brief Options for writing a dataset.
struct ARROW_DS_EXPORT FileSystemDatasetWriteOptions {
.
.
.
/// Template string used to generate fragment basenames.
/// {i} will be replaced by an auto incremented integer.
std::string basename_template;
.
.
.
}
```
If there is any method to specify the filename when none partition?

### Component(s)

Java

貢獻指南

開啟貢獻指南

研究方向

從 Java DatasetFileWriter::write 和對 writeFromScannerToFile 的 JNI 呼叫開始,接著檢查 C++ FileSystemDatasetWriteOptions::basename_template。判斷非分割寫入是否能接受不含 {i} 的確切檔名;當已確立支援的行為,且 Java/C++ 路徑能處理該情況或清楚記錄該情況時,即表示完成。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
cpp, java
領域
data-engineering
Issue 類型
功能
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。