apache / apache/accumulo

Call hsync(), if supported, immediately prior to closing DFS compaction files.

Open
#4,455 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

According to [this documentation](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/outputstream.html#HDFS_and_OutputStream.close.28.29) its best to call hsync() prior to close. This could possibly be done for files created for minor and major compaction in Accumulo. For compactions could call StreamCapabilities to see if hsync() is supported and if its then call it immediately before calling close.

Contributor guide

Open the contributing guide

Research direction

Start by locating where minor and major compaction files are created and closed, then read the referenced output-stream documentation and the StreamCapabilities API. Done means each compaction stream checks whether hsync() is supported and calls it immediately before close.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.