apache / apache/parquet-java

Only call hsync() and hflush() on supported filesystems

Open
#2,990 0 comments 0 reactions 0 assignees View on GitHub
Type: enhancement
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

### Describe the enhancement requested

Instead of log the unsupported call check capabilities and call only on supported filesystems - e.g. S3A does not.

Write in a ParquetFile on S3 calls on hsync() or hflush() are logged: Application invoked the Syncable API against stream writing

### Comments

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the ParquetFile write path where hsync() and hflush() are invoked, and inspect how filesystem capabilities are exposed. Done means unsupported filesystems such as S3A no longer log Syncable warnings, while supported filesystems continue to receive the calls.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
backend, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.