alibaba / alibaba/GraphScope

Proposal of Graph File Format `Reader` and `Writer` APIs

Open
#1,899 0 comments 0 reactions 1 assignee Claimed by @acezen View on GitHub
Dominant language
C++
Stars
3.6k
Forks
468
Avg merge
29m
Merged PRs (30d)
1

Description

### Reader
```C++
# create a reader with graph meta yaml
void Init(FILE meta_file);

# The graph meta API
int VerticesNum(vertex_label_name);
int EdgeNum(edge_label_name);
int TotalVerticeNum();
int TotalEdgeNum();
int VertexLabelNum();
int EdgeLabelNum();

# Get the chunk number of certain vertex or edge
int GetVertexChunkNumber(vertex_label);
int GetEdgeChunkNumber(src_label, edge_label, dst_label);

# Get the certain chunk of vertice as stream to read
Stream GetVertice(int label, int chunk_offset);
Stream GetVerticeProperty(label, property, chunk_offset);
Stream GetVertexProperty(global_vertex_id);
# Get the certain chunk of edges as stream to read
# 三元组 (src_label, e_label, dst_label) 定义一种边
Stream GetEdges((src_label, e_label, dst_label), int chunk_offset);
Stream GetEdgesProperty((src_label,e_label, dst_label), int chunk_offset);

# 存在offset表的情况, 通过CSR结构读取点相关的边
Stream GetOffsets((src_label,e_label, dst_label), int chunk_offset);
# Get the edges of certain vertex
Stream GetEdges(global_vertex_id, (src_label,e_label, dst_label));
Stream GetEdgeProperties(global_vertex_id, (src_label,e_label, dst_label));

# TODO: 考虑外部点,跨分区边
```

### Writer
```C++
void Init(path_prefix, comm_spec);

# set some graph meta message
void Set(vertex_label_num, edge_label_num, list, list,
total_vertex_num, total_edge_num, vertex_chunk_size, edge_chunk_size);

# maping global vertex id in writer, dumps the vertices to chunks, and gather a global vertex mapping
Status DumpVertices(List);

# dumps the properties to chunks base on the global vertex mapping
Status DumpVertexProperties(PropertyTable, vertex_label);

# dumps the edges to chunks base on the global vertex mapping
Status DumpEdges(EdgesTable, (src_label, edge_label, dst_label));

# dumps the edges property to chunks base on the global vertex mapping
Status DumpEdgeProperty(EdgesPropertyTable, (src_label, edge_label, dst_label));

# TODO: 考虑shuffle
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.