apache / apache/accumulo

Improve log recovery times

Open
#4,505 3 comments 0 reactions 1 assignee Claimed by @keith-turner View on GitHub
enhancement
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

**Is your feature request related to a problem? Please describe.**

Write ahead log recovery can take a while because of the following two behaviors.

* Tablet servers processes only do a single log recovery at time
* All tablets, even if they have no data in the write ahead log, will go through the log recovery process when being loaded on a tablet.

Those behaviors make log recovery times correlate with the number of tablets per tserver. So as the number of tablets per tserver increases, log recovery time increases.

**Describe the solution you'd like**

Allow parallel log recovery and faster log recovery. The parallelism is related to #4429, but that change does not completely solve the issue as the lock is still acquired for log recovery.

* Use a cache during log recovery when reading from sorted walog rfiles
* Inspect tablet w/ logs before acquiring recovery lock to see if they contain data

**Describe alternatives you've considered**

Could potentially produce an F file for log recovery outside of the tablet server somewhere (similar to external compactions). This may have been discussed on an elasticity related issue, but could not find it. This would be a much larger change and probably would be suitable to do in 2.1. It may require completly refactoring the tablet minor compaction code to make it usable elsewhere.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.