Look into using a Striped lock for bulk load zookeeper checks.
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 487
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 13
Description
Bulk load RPCs check to see if the transaction is still alive in ZooKeeper before executing. While doing this check a tablet server wide [lock is held](https://github.com/apache/accumulo/blob/11e26d481b7885e46ce439e1345f0bf9bdd5dfb2/server/base/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java#L126). Therefore concurrent bulk loads executing different transactions may have to wait on each other. This could be avoided by using a [Striped] lock.
It's possible that allowing more concurrency may not be a good thing in that it could place more load on zookeeper. Could possibly make the stripe size small to allow a small amount of concurrency.
[Striped]: https://static.javadoc.io/com.google.guava/guava/27.0.1-jre/com/google/common/util/concurrent/Striped.html
Contributor guide
Research direction
Read server/base/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java at the linked lock location, then review the closed pull request #1710. Check how bulk-load transaction checks are synchronized and whether striped locking changes concurrency without imposing unacceptable ZooKeeper load; done means the approach is evaluated and its behavior is validated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100