apache / apache/incubator-graphar

[Feat][Doc] Benchmarking C++/Spark readers with/without filter pushdown

Open
#403 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
373
Forks
93
Avg merge
7d 21h
Merged PRs (30d)
7

Description

**Is your feature request related to a problem? Please describe.**
Filter pushdown is a sophisticated feature available with C++/Spark readers that has the potential to enhance query performance. It's important to evaluate its effectiveness.

**Describe the solution you'd like**
To gauge the impact of filter pushdown, I propose using the LDBC dataset to benchmark the performance of reading operations. Specifically, we can measure how efficiently the C++/Spark readers can filter vertices or edges with certain property conditions when filter pushdown is enabled compared to when it is not.

**Additional context**
This request is in continuation of the discussion in issue #389

Contributor guide

Open the contributing guide

Research direction

Start by reading issue #389 and locating the C++ and Spark reader entry points involved in filter pushdown. Run comparable LDBC vertex and edge reading benchmarks with pushdown enabled and disabled, then document the performance results and comparison.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, spark
Domain
data-engineering, performance
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.