apache / apache/incubator-graphar

[FEATURE][Java] Add the GraphAr write facade

Open
#973 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
373
Forks
93
Avg merge
7d 21h
Merged PRs (30d)
7

Description

### Describe the enhancement requested

Add the GraphAr write facade that turns a stream of edges into a published GraphAr dataset: vertex and edge chunk files, ordered adjacency, offset chunks, and the metadata that describes them. This is the layer directly above `graphar-core` (#967) and the physical write contract in `graphar-io-api` (#959), and it is what makes a Java-produced dataset readable by the C++ and Spark implementations.

Component(s): Java

## Scope

- Add `graphar-writer` with a `GraphWriter` facade that accepts edge records and emits a complete GraphAr dataset through the existing `PhysicalWriter` contract.
- Cover edge write options (chunk size, ordering, adjacency list type) and reported write statistics.
- Keep the writer format-neutral: it must talk to `graphar-io-api` only, never to a concrete format module.
- Add unit tests for chunk boundary emission, offset chunk correctness, and rejected option combinations.

## Non-goals

No new format backend, no Arrow dependency, no reader facade, no query semantics, and no changes to the physical write contract itself.

## Compatibility and acceptance

Output must be byte-compatible with the canonical GraphAr layout: a dataset written by this module loads in the existing implementations without conversion. Java 11 compatible.

## Dependency order

This slice depends on `graphar-core` (#967, PR #968) and on a merged physical format backend (#964) to have anything to write through. It should land after both.

Contributor guide

Open the contributing guide

Research direction

Start by reading the graphar-core work in #967, the PhysicalWriter contract in graphar-io-api from #959, and the merged backend dependency in #964. Add the graphar-writer module and GraphWriter facade, then add unit tests for chunk boundaries, offset chunks, and rejected option combinations. Done means Java 11-compatible output is byte-compatible with the canonical GraphAr layout and readable by the existing implementations.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.