JanusGraph / JanusGraph/janusgraph

Remove dependency on live hbase when using HBaseSnapshotInputFormat

Open
#838 0 comments 0 reactions 0 assignees View on GitHub
storage/hbase
Dominant language
Java
Stars
5.8k
Forks
1.2k
Avg merge
13h 53m
Merged PRs (30d)
6

Description

This is a follow-up issue after #656.

The current JanusGraph input format base implementation needs to go to a live backend to get schema data. This is ok, particularly for input formats that operate on live backend, e.g. CassandraInputFormat and HBaseInputFormat, because we scan the table on live backend anyway.

But HBaseSnapshotInputFormat works on a snapshot, which is static data on a file system. For example, it can be an exported snapshot stored on S3 without a live HBase running. It is self-contained with all the table data for a graph. We should be able to get the schema data from the static snapshot in addition to the vertices and edge data. Then we will be able to work on a snapshot independently.

Contributor guide

Open the contributing guide

Research direction

Start with the JanusGraph input format base implementation and HBaseSnapshotInputFormat, tracing how schema data is currently obtained from a live backend. The work is complete when schema, vertex, and edge data can all be read from a static HBase snapshot, including one stored on S3, without a running HBase instance.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.