InternalParquetRecordWriter flushRowGroupToStore not actually writing to file
- 主要语言
- Java
- 星标
- 3.1k
- 派生
- 1.6k
- 平均合并
- 3 天 12 小时
- 30 天内合并 PR
- 33
描述
### Describe the bug, including details regarding any error messages, version, and platform.
Description:
The flushRowGroupToStore method in InternalParquetRecordWriter purports to "Flush mem columnStore to file," but no data is actually written to the file system until the close method is invoked. This behavior is unexpected, as one would anticipate data being flushed.
Expected Result:
Data should be written to the file when flushRowGroupToStore is invoked, effectively flushing the in-memory column store contents to the file.
Actual Result:
File contents remain empty until the close method is called, where parquetFileWriter.end is invoked to write the data.
Version : parquet-hadoop 1.13.1
### Component(s)
_No response_
贡献指南
这个仓库没有索引到贡献指南
调研方向
首先阅读 InternalParquetRecordWriter,尤其是 flushRowGroupToStore 和 close,然后跟踪 parquetFileWriter.end 如何写入缓冲的数据。确认预期的 flush 边界,并添加或更新对 flushRowGroupToStore 之后文件内容的覆盖;当 close 被调用之前文件已包含被 flush 的 row group 时,即表示完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java
- 领域
- data
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 45/100