apache / apache/incubator-graphar
OSPP24 Idea: Implement ETL CLI Tools for GraphAr
- Dominant language
- C++
- Stars
- 373
- Forks
- 93
- Avg merge
- 7d 21h
- Merged PRs (30d)
- 7
Description
### Describe the enhancement requested
### Description
GraphAr is designed as a unified storage format for graph data, aiming to provide a standardized graph data storage format for easy import/export, as well as exchange and sharing of graph data.Beyond the foundational format design, GraphAr currently also offers libraries in C++, Java, Python, and Scala to enable users to work with GraphAr formatted data across different programming environments.
To facilitate the use of GraphAr formatted data, we aim to provide a command-line tool based on these libraries. This tool will be used for converting data from various sources into GraphAr formatted data and vice versa - transforming GraphAr formatted data into other formats.
This command-line tool needs to support the following features:
- A user-friendly command-line interface
- Graph data management: Users can use the CLI tool to view basic information about the GraphAr formatted data, such as the number of nodes, edges, properties, and related Schema information.
- Graph data import: Users can import data from other formats into GraphAr format through the CLI tool.
- Support for importing large-scale data: Users can use the CLI tool to import massive datasets into GraphAr format.
- (Optional) Graph data export: Users can export GraphAr formatted data into other formats using the CLI tool (lower priority).
### Deliverables
1. A CLI tool that meets the above requirements
2. Detailed design and usage documentation
### Component(s)
Other
### Reference
- [Neo4j ETL Command-line tool](https://neo4j.com/labs/etl-tool/1.5.0/neo4j-etl/)
- [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/)
- [Click](https://click.palletsprojects.com/en/8.1.x/)
Contributor guide
Assessment
This issue has not been assessed yet.