apache / apache/parquet-java

InternalParquetRecordWriter flushRowGroupToStore not actually writing to file

Open
#3,171 5 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Description:
The flushRowGroupToStore method in InternalParquetRecordWriter purports to "Flush mem columnStore to file," but no data is actually written to the file system until the close method is invoked. This behavior is unexpected, as one would anticipate data being flushed.

Expected Result:
Data should be written to the file when flushRowGroupToStore is invoked, effectively flushing the in-memory column store contents to the file.

Actual Result:
File contents remain empty until the close method is called, where parquetFileWriter.end is invoked to write the data.

Version : parquet-hadoop 1.13.1

### Component(s)

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.