apache / apache/accumulo

Look into using a Striped lock for bulk load zookeeper checks.

Open
#873 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

Bulk load RPCs check to see if the transaction is still alive in ZooKeeper before executing. While doing this check a tablet server wide [lock is held](https://github.com/apache/accumulo/blob/11e26d481b7885e46ce439e1345f0bf9bdd5dfb2/server/base/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java#L126). Therefore concurrent bulk loads executing different transactions may have to wait on each other. This could be avoided by using a [Striped] lock.

It's possible that allowing more concurrency may not be a good thing in that it could place more load on zookeeper. Could possibly make the stripe size small to allow a small amount of concurrency.

[Striped]: https://static.javadoc.io/com.google.guava/guava/27.0.1-jre/com/google/common/util/concurrent/Striped.html

Contributor guide

Open the contributing guide

Research direction

Read server/base/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java at the linked lock location, then review the closed pull request #1710. Check how bulk-load transaction checks are synchronized and whether striped locking changes concurrency without imposing unacceptable ZooKeeper load; done means the approach is evaluated and its behavior is validated.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.