apache / apache/age

Allow load_labels_from_file and load_edges_from_file to accept file data directly

Open
#432 4 comments 4 reactions 0 assignees View on GitHub
enhancement
Dominant language
C
Stars
4.8k
Forks
523
Avg merge
1d 2h
Merged PRs (30d)
9

Description

It seems that there is a limit to the quantity of inserts using 'CREATE' in cypher with PG due to the nature of the query. Most importantly creating and inserting a large quantity of nodes when generated by reading a csv file client side with CREATE queries is much slower than utilizing load_labels_from_file and load_edges_from_file functions implemented as a part of the extension.

I understand that from the age documentation when creating multiple vertices we can create a query like so
**

> SELECT *
> FROM cypher('graph_name', $$
> CREATE (n), (m), (e), ...
> $$) as (v agtype);

**
when creating any number of nodes over 1664 an error is thrown by PG stating
“target lists can have at most 1664 entries” error code 54011. The solution in this case is to create individual queries for each node. Individual create queries for each node take a very long time to process by the backend. Whereas the functions stated above "load_labels_from_file" e.g. is much faster but only works with a direct path to a file.

Possible Solution:
Implement load_labels_from_file and load_edges_from_file to accept a sequence of bytes rather than just a file path to parse csv formatted data. This will allow quick client side initialization of data from any client side application.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.