Allow load_labels_from_file and load_edges_from_file to accept file data directly
- Dominant language
- C
- Stars
- 4.8k
- Forks
- 523
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 9
Description
It seems that there is a limit to the quantity of inserts using 'CREATE' in cypher with PG due to the nature of the query. Most importantly creating and inserting a large quantity of nodes when generated by reading a csv file client side with CREATE queries is much slower than utilizing load_labels_from_file and load_edges_from_file functions implemented as a part of the extension.
I understand that from the age documentation when creating multiple vertices we can create a query like so
**
> SELECT *
> FROM cypher('graph_name', $$
> CREATE (n), (m), (e), ...
> $$) as (v agtype);
**
when creating any number of nodes over 1664 an error is thrown by PG stating
“target lists can have at most 1664 entries” error code 54011. The solution in this case is to create individual queries for each node. Individual create queries for each node take a very long time to process by the backend. Whereas the functions stated above "load_labels_from_file" e.g. is much faster but only works with a direct path to a file.
Possible Solution:
Implement load_labels_from_file and load_edges_from_file to accept a sequence of bytes rather than just a file path to parse csv formatted data. This will allow quick client side initialization of data from any client side application.
Contributor guide
Assessment
This issue has not been assessed yet.