JanusGraph / JanusGraph/janusgraph
Remove dependency on live hbase when using HBaseSnapshotInputFormat
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
This is a follow-up issue after #656.
The current JanusGraph input format base implementation needs to go to a live backend to get schema data. This is ok, particularly for input formats that operate on live backend, e.g. CassandraInputFormat and HBaseInputFormat, because we scan the table on live backend anyway.
But HBaseSnapshotInputFormat works on a snapshot, which is static data on a file system. For example, it can be an exported snapshot stored on S3 without a live HBase running. It is self-contained with all the table data for a graph. We should be able to get the schema data from the static snapshot in addition to the vertices and edge data. Then we will be able to work on a snapshot independently.
Contributor guide
Research direction
Start with the JanusGraph input format base implementation and HBaseSnapshotInputFormat, tracing how schema data is currently obtained from a live backend. The work is complete when schema, vertex, and edge data can all be read from a static HBase snapshot, including one stored on S3, without a running HBase instance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100