apache / apache/accumulo

WalStateManager does not see a consistent snapshot of WAL state in ZK (race condition)

Open
#1,557 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

WalStateManager does multiple ZooKeeper lookups to gather the state of the WALs. This can lead to race conditions, although it is not clear to what extent these matter. This should be investigated further.

`WalStateManager.getAllMarkers()` does one ZK lookup for the children of `/wals`, which gets the list of tservers, then loops through that and gets the WALs under each of those with separate ZK lookups, and then `state()` does additional ZK lookups to get the state of each WAL marker. All of these separate lookups can lead to a very inconsistent view of the WAL states. But... does this matter?

Contributor guide

Open the contributing guide

Research direction

Start by reading WalStateManager.getAllMarkers() and state(), focusing on the ZooKeeper lookups for /wals, tablet servers, WALs, and WAL markers. Determine whether the separate lookups can produce a meaningfully inconsistent WAL view and define what investigation or change is needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.