Write a tool to find out the actual point at which MapReduce becomes better than Fluo.
- Dominant language
- Java
- Stars
- 200
- Forks
- 73
- PR merge metrics
- No merged PRs in 30d
Description
One of the benchmarks done in the Percolator paper show that larger crawl rates (changes to 40% or more of the repository) are better for mapreduce indexing and less than 40% is better for Percolator. I'd like to see the same numbers for Fluo and Accumulo. There was a drastic point of exponential growth when the crawl rate went above the point where mapreduce is useful. Would our numbers look the same? Would we be able to support better crawl rates in Fluo? Can we strive to make this better over time?
Specifically, this tool can be used as we develop Accumulo and Fluo to
1) initially see what our numbers look like compared to Percolator proper, and
2) see if our numbers have changed (for better or for worse) as we continue to add new features and optimize
Contributor guide
Assessment
This issue has not been assessed yet.