apache / apache/accumulo-fluo

Batch reads by tablet server

Open
#138 0 comments 0 reactions 0 assignees View on GitHub
wish
Dominant language
Java
Stars
200
Forks
73
PR merge metrics
No merged PRs in 30d

Description

Currently each thread that reads from a tablet server will use its own Accumulo Scanner or BatchScanner. This will result in RPC calls specifically for the thread. Below are some reasons a shared batch scanner does not work really well :
- The same iterator configuration must be used for request to a batch scanner. Since each fluo thread is executing different phases of 2PC and using different timestamps, each threads will need different iterators and/or different iterator configurations.
- The batch scanner does not aggregate request from multiple user threads. A batch scanner can be used by multiple threads, but its tricky. Need to synchronize configuration and iterator acquisition. After that an iterator will scan all of the ranges passed in using a shared thread pool, but will not aggregate ranges from other from multiple request over same RPCs.
- There is also no mechanism for multiple threads to submit ranges and have the results grouped by range and requesting thread.

Ideally there would be a batch scanner that behaved like the new conditional writer. The conditional writer meets all of these criteria for its purpose.

Contributor guide

Open the contributing guide

Research direction

Start by reading the issue's requirements for batching reads from a tablet server and compare them with the existing conditional writer behavior. No files or tests are named; done would require a design and implementation that aggregates requests while preserving per-thread ranges, iterator configurations, timestamps, and result grouping.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.