JanusGraph / JanusGraph/janusgraph
graph creation with default id authority wait time is too slow for use in unit tests with multiple repeated graph creations
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
For confirmed bugs, please report:
- Version: org.janusgraph:janusgraph-core:0.4.0
- Storage Backend: inmemory
- Mixed Index Backend: No
- Mailing list Thread URL: https://groups.google.com/forum/#!topic/janusgraph-users/ME9U6-0Xn1Y
- Steps to Reproduce:
I use inmemory JanusGraph storage to run my tests. Creation of JanusGraph is very slow and it seems because it waits for some period of time for new id blocks.
The sample below compares creating inmemory JanusGraph with automatic and custom id generation. It shows the strange behavior that the time of JanusGraph creation does not depend on properties ids.authority.wait-time, ids.renew-timeout, ids.block-size, and ids.renew-percentage but does depend on order of JanusGraph instances creation.
See the code example below.
If I first run creating vertices with generated ids and then custom ids
I got:
[generated ids] vertices: 1000, elapsed time: 1255(ms), graph
creation time: 593
[custom ids] vertices: 1000, elapsed time: 255(ms), graph creation
time: 4
The method with generated ids takes about 1 sec and the graph creation
takes about 600ms.
If I first run creating vertices with custom ids I got:
[custom ids] vertices: 1000, elapsed time: 1125(ms), graph creation
time: 590
[generated ids] vertices: 1000, elapsed time: 401(ms), graph creation
time: 3
The method with custom ids takes about 1sec and graph creation takes
about 600ms.
If I run only one of them then each takes about 1 sec and graph creation
takes about 600ms.
```java
import
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
import org.apache.tinkerpop.gremlin.structure.T;
import org.janusgraph.core.JanusGraph;
import org.janusgraph.core.JanusGraphFactory;
import org.janusgraph.core.JanusGraphTransaction;
import org.janusgraph.graphdb.database.StandardJanusGraph;
import org.janusgraph.graphdb.idmanagement.IDManager;
public class JanusGraphSample {
private static final String LABEL = "SampleLabel";
private static final String KEY = "SampleKey";
private static final String VALUE = "SampleValue";
private static final int N = 1000;
public static void main(String[] args) {
// Try to run these methods in reverse order
addVerticesWithGeneratedIds();
addVerticesWithCustomIds();
}
private static JanusGraph getJanusGraph(boolean customIds) {
JanusGraphFactory.Builder builder = JanusGraphFactory.build()
.set("storage.backend", "inmemory")
.set("ids.authority.wait-time", "5")
.set("ids.renew-timeout", "50")
.set("ids.block-size", "1000000000")
.set("ids.renew-percentage", "0.3");
if (customIds) {
builder = builder.set("graph.set-vertex-id", "true");
}
return builder.open();
}
static long verticesCount(JanusGraph graph) {
try (JanusGraphTransaction tx = graph.newTransaction()) {
GraphTraversalSource g = tx.traversal();
return g.V().count().next();
}
}
public static void addVerticesWithGeneratedIds() {
long time = System.currentTimeMillis();
try (JanusGraph graph = getJanusGraph(false)) {
long creationTime = System.currentTimeMillis();
try (JanusGraphTransaction tx = graph.newTransaction()) {
GraphTraversalSource g = tx.traversal();
for (int i = 1; i <= N; i++) {
String value = String.format("%s-%d", VALUE, i);
g.addV(LABEL).property(KEY, value).next();
}
tx.commit();
}
long elapsedTime = System.currentTimeMillis();
long vertices = verticesCount(graph);
System.out.printf("[generated ids] vertices: %d, elapsed
time: %d(ms), graph creation time: %d%n", vertices, elapsedTime - time,
creationTime - time);
}
}
public static void addVerticesWithCustomIds() {
long time = System.currentTimeMillis();
try (JanusGraph graph = getJanusGraph(true)) {
long creationTime = System.currentTimeMillis();
IDManager idManager = ((StandardJanusGraph)
graph).getIDManager();
try (JanusGraphTransaction tx = graph.newTransaction()) {
GraphTraversalSource g = tx.traversal();
for (int i = 1; i <= N; i++) {
long id = idManager.toVertexId(i);
String value = String.format("%s-%d", VALUE, i);
g.addV(LABEL).property(T.id, id).property(KEY,
value).next();
}
tx.commit();
}
long elapsedTime = System.currentTimeMillis();
long vertices = verticesCount(graph);
System.out.printf("[custom ids] vertices: %d, elapsed time:
%d(ms), graph creation time: %d%n", vertices, elapsedTime - time,
creationTime - time);
}
}
}
```
Contributor guide
Research direction
Start by reproducing the sample through getJanusGraph, addVerticesWithGeneratedIds, and addVerticesWithCustomIds using the inmemory backend, varying creation order and the listed ID settings. Trace JanusGraphFactory.Builder.open and the IDManager behavior to identify the unnecessary wait; done means repeated graph creation avoids the order-dependent delay without changing the reported ID behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100