apache / apache/incubator-graphar
[Feat][Doc] Benchmarking C++/Spark readers with/without filter pushdown
- Dominant language
- C++
- Stars
- 373
- Forks
- 93
- Avg merge
- 7d 21h
- Merged PRs (30d)
- 7
Description
**Is your feature request related to a problem? Please describe.**
Filter pushdown is a sophisticated feature available with C++/Spark readers that has the potential to enhance query performance. It's important to evaluate its effectiveness.
**Describe the solution you'd like**
To gauge the impact of filter pushdown, I propose using the LDBC dataset to benchmark the performance of reading operations. Specifically, we can measure how efficiently the C++/Spark readers can filter vertices or edges with certain property conditions when filter pushdown is enabled compared to when it is not.
**Additional context**
This request is in continuation of the discussion in issue #389
Contributor guide
Research direction
Start by reading issue #389 and locating the C++ and Spark reader entry points involved in filter pushdown. Run comparable LDBC vertex and edge reading benchmarks with pushdown enabled and disabled, then document the performance results and comparison.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, spark
- Domain
- data-engineering, performance
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100