Batch reads by tablet server
- Dominant language
- Java
- Stars
- 200
- Forks
- 73
- PR merge metrics
- No merged PRs in 30d
Description
Currently each thread that reads from a tablet server will use its own Accumulo Scanner or BatchScanner. This will result in RPC calls specifically for the thread. Below are some reasons a shared batch scanner does not work really well :
- The same iterator configuration must be used for request to a batch scanner. Since each fluo thread is executing different phases of 2PC and using different timestamps, each threads will need different iterators and/or different iterator configurations.
- The batch scanner does not aggregate request from multiple user threads. A batch scanner can be used by multiple threads, but its tricky. Need to synchronize configuration and iterator acquisition. After that an iterator will scan all of the ranges passed in using a shared thread pool, but will not aggregate ranges from other from multiple request over same RPCs.
- There is also no mechanism for multiple threads to submit ranges and have the results grouped by range and requesting thread.
Ideally there would be a batch scanner that behaved like the new conditional writer. The conditional writer meets all of these criteria for its purpose.
Contributor guide
Research direction
Start by reading the issue's requirements for batching reads from a tablet server and compare them with the existing conditional writer behavior. No files or tests are named; done would require a design and implementation that aggregates requests while preserving per-thread ranges, iterator configurations, timestamps, and result grouping.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100