[proposal] Python API enhancement for compatibility of Giraph
- Dominant language
- C++
- Stars
- 3.6k
- Forks
- 468
- Avg merge
- 29m
- Merged PRs (30d)
- 1
Description
Here is a revision proposal of the Python API, to
- support the Java Giraph apps, by introducing the capability of the customized input/output parser and formats.
- hide the concept of `context`, by always generating a new graph after an application ran.
```python
# class path
# upload the jars/gars to k8s
sess.add_lib("xxx.jar")
# Loading graph
# Leave add_vertices and add_edges for property graphs
# Use load_from? e.g.,
# formatter parser:
# giraph:com.p2pvif
# gar:xyb.gar:ClassName
g1 = sess.load_from(vertices="p2p.v", vformat="giraph:com.P2PVIF", edges="p2p.e", eformat="giraph:com.P2PEIF")
# vertices can be omitted when only have efile.
g2 = sess.load_from(edges="p2p.e", eformat="giraph:com.P2PEIF")
# adj format stores edges along with the vertices.
g3 = sess.load_from(vertices="p2p.v", vformat="giraph:com.P2PADJ")
# enum class Fmt provides many built-in formatters TODO
g4 = sess.load_from(vertices="p2p.v", vformat=Fmt.ADJVidFloatVidInt)
# scanf formatter TODO
g5 = sess.load_from(vertices="p2p.v", vformat="%d %d", edges="p2p.e", eformat="{src:int} {dst:int} {name:string}")
# computation
giraph_sssp = load_app(algo="giraph:com.alibaba.graphscope.example.bfs.BFS")
# loaded graph can be directed used for the giraph app
# an implicit conversion triggers if g is a property graph.
# for apps, always return a new graph g
new_graph = giraph_sssp(g1, src=6)
partition_app_name = load_app("gar_partitioner:xxx.gar:xxx")
# hence partitioner is a special kind of app.
g6 = partition_app_name(g5, partitioner="2d")
g7 = partition_app_name(g6, partitioner="HashingPartitioner")
# move to_xxx methods to graph, rather than context.
new_graph.to_numpy('v.data')
# hence the selector can be used to output the graph elements.
new_graph.to_numpy('v.id')
# no longer has r selector
# for some apps return void(does not modify the graph)
# or generate tensors, keep the context and the r selector
ctx = pattern_match(g, pattern="xxx")
ctx.to_tensor("")
# unify the output method to existing save_to
# existing method for serialization
new_graph.save_to("file://serialization_path")
# to output with a customized formatter
new_graph.save_to("file:///tmp/path", format="giraph:xxx")
```
Contributor guide
Assessment
This issue has not been assessed yet.