apache / apache/incubator-graphar

[Feat] Integrate LDBC spark connector as input data source for GraphAr spark tool

Open
#35 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
373
Forks
93
Avg merge
7d 21h
Merged PRs (30d)
7

Description

**Is your feature request related to a problem? Please describe.**
LDBC provides a synthetic graph generator running on Spark (https://github.com/ldbc/ldbc_snb_datagen_spark). We can utilize the GraphAr spark library to integrate with this graph generator, for dumping the generated graph data into GraphAr files.

**Describe the solution you'd like**
Refer to the API reference of Reader/Writer and graph-level interface of the GraphAr Spark library. The integration with neo4j Spark connector (#107 ) can also help.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

Research direction

Start with the GraphAr Spark library's Reader/Writer API reference and graph-level interface, then review the Neo4j Spark connector integration in issue #107. The feature is done when LDBC's Spark-generated graph data can be dumped into GraphAr files through the Spark tool.

Written by the indexing model from the issue text.

Assessment

Tech stack
spark
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.