apache / apache/incubator-graphar

OSPP24 Idea: Implement ETL CLI Tools for GraphAr

Open
#463 3 comments 4 reactions 1 assignee Claimed by @jiajunly View on GitHub
enhancement
Dominant language
C++
Stars
373
Forks
93
Avg merge
7d 21h
Merged PRs (30d)
7

Description

### Describe the enhancement requested

### Description

GraphAr is designed as a unified storage format for graph data, aiming to provide a standardized graph data storage format for easy import/export, as well as exchange and sharing of graph data.Beyond the foundational format design, GraphAr currently also offers libraries in C++, Java, Python, and Scala to enable users to work with GraphAr formatted data across different programming environments.

To facilitate the use of GraphAr formatted data, we aim to provide a command-line tool based on these libraries. This tool will be used for converting data from various sources into GraphAr formatted data and vice versa - transforming GraphAr formatted data into other formats.

This command-line tool needs to support the following features:

- A user-friendly command-line interface
- Graph data management: Users can use the CLI tool to view basic information about the GraphAr formatted data, such as the number of nodes, edges, properties, and related Schema information.
- Graph data import: Users can import data from other formats into GraphAr format through the CLI tool.
- Support for importing large-scale data: Users can use the CLI tool to import massive datasets into GraphAr format.
- (Optional) Graph data export: Users can export GraphAr formatted data into other formats using the CLI tool (lower priority).

### Deliverables
1. A CLI tool that meets the above requirements
2. Detailed design and usage documentation

### Component(s)

Other

### Reference
- [Neo4j ETL Command-line tool](https://neo4j.com/labs/etl-tool/1.5.0/neo4j-etl/)
- [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/)
- [Click](https://click.palletsprojects.com/en/8.1.x/)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.