apache / apache/accumulo

Add RFile block info to validate the block was read correctly

Open
#3,698 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

Ideas for this:

* Add magic number at the beginning of each RFile block, to account for empty blocks
* Add hash at the end of each RFile block to ensure the block is fully read and doesn't return early, and to ensure there aren't any errors in the block that was just read (may particularly benefit with block caching, which forces blocks to be fully read)

Contributor guide

Open the contributing guide

Research direction

Start by locating the RFile block read and write entry points, including the block-caching path mentioned in the issue. Determine how block boundaries and empty blocks are represented before assessing magic-number and end-of-block hash validation. Done means blocks can be checked for complete reads and detected errors without returning early.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.