Add Linked data benchmarks
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Recently I came across LDBC benchmarks which is focused on graph-like workloads. I'm wondering whether Datafusion already covers the features the queries need. While I don't think it's as important as TPC-H it'd increase the coverage helping to identify performance regressions during the Datafusion development. This would be an extra tool to get a broader picture in a structured way (at least more structured than ad-hoc queries)
**Describe the solution you'd like**
Supporting the queries written for PostgreSQL: https://github.com/ldbc/ldbc_snb_bi/tree/main/postgres/queries .
**Describe alternatives you've considered**
Not implementing it. Optimizing Datafusion to perform well on this particular benchmark is out of the scope as well. My assumption is that OLAP should be first-class and this should be a second class target.
**Additional context**
While it's not an OLAP workload, I believe Datafusion would perform relatively or extremely well.
Cc @Dandandan IIRC you contributed the most (CTE+UNION ALL) in this field
Contributor guide
Research direction
Start by reviewing the PostgreSQL queries in the linked LDBC SNB BI repository and checking which DataFusion SQL features they require, especially CTE and UNION ALL support. Confirm the query set can run in DataFusion and define benchmark coverage that can reveal performance regressions without optimizing specifically for this workload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, rust
- Domain
- databases, performance, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100