apache / apache/accumulo

Support splitting tablet with lots of files

Open
#5,100 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

**Is your feature request related to a problem? Please describe.**

Currently in the main branch for Accumulo 4.0 there is no support for splitting tablets with lots of files. This exist in earlier versions of Accumulo but was not carried forward. Need to readd support for this.

**Describe the solution you'd like**

In earlier version of accumulo when a tablet had lots of files the split code would merge the rfile indexes in batches and then iterated over this merged view. For example if a tablet had 1000 files it might read 200 files at time merging their indexes into a new file, then it would read over the intermediate merged index files to compute the split point.

Contributor guide

Open the contributing guide

Research direction

Compare the Accumulo 4.0 tablet split code with the earlier version that handled tablets containing many files. Focus on batching rfile indexes into intermediate merged views before computing the split point. Done means a tablet with many files can be split successfully without requiring all indexes to be merged at once.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.