JanusGraph / JanusGraph/janusgraph

Distributed reindexing for any storage backend

Open
#847 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
5.8k
Forks
1.2k
Avg merge
13h 53m
Merged PRs (30d)
6

Description

Maybe the title is not the best but I have noticed one disturbing thing: MapReduceIndexManagement class has direct dependencies on the concrete classes for HBase AND Cassandra. I see two issues with this: first, the dependencies themselves, the second - the lack of support for any other storage backends. For example, AWS DynamoDB.

It seems that, at least, janusgraph-hadoop should be backend-independent and something like janusgraph-hadoop-cassandra and janusgraph-hadoop-hbase should be created to deal with backend-specific implementations. I suspect this may require some additional layer to implement the backend-independent partititione, scanner and input formats.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the MapReduceIndexManagement class and the janusgraph-hadoop module to map its direct HBase and Cassandra dependencies. Determine the boundaries needed for backend-independent partitioning, scanning, and input formats, then define backend-specific implementations such as janusgraph-hadoop-cassandra and janusgraph-hadoop-hbase; done means another storage backend can be supported without coupling the core module to concrete backends.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, cassandra, java
Domain
data-engineering, databases, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.