apache / apache/parquet-java

Properly close buffers

Open
#3,558 0 comments 0 reactions 0 assignees View on GitHub
Type: enhancement
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

### Describe the enhancement requested

As @pan3793 pointed out https://github.com/apache/parquet-java/pull/3360#issuecomment-3588399639, when updating to a later version of Hadoop, it starts failing with an error of lingering buffers.

### Component(s)

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the discussion in Apache Parquet PR 3360, especially the referenced comment, and reproduce the lingering-buffer failure after updating Hadoop. Locate the buffer lifecycle involved in that failure; done means buffers are properly closed and the later Hadoop version no longer reports lingering buffers.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.