Support splitting tablet with lots of files
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 487
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 13
Description
**Is your feature request related to a problem? Please describe.**
Currently in the main branch for Accumulo 4.0 there is no support for splitting tablets with lots of files. This exist in earlier versions of Accumulo but was not carried forward. Need to readd support for this.
**Describe the solution you'd like**
In earlier version of accumulo when a tablet had lots of files the split code would merge the rfile indexes in batches and then iterated over this merged view. For example if a tablet had 1000 files it might read 200 files at time merging their indexes into a new file, then it would read over the intermediate merged index files to compute the split point.
Contributor guide
Research direction
Compare the Accumulo 4.0 tablet split code with the earlier version that handled tablets containing many files. Focus on batching rfile indexes into intermediate merged views before computing the split point. Done means a tablet with many files can be split successfully without requiring all indexes to be merged at once.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100