JanusGraph / JanusGraph/janusgraph

Enable serialization of JanusGraph element types

Open
#1,519 0 comments 0 reactions 0 assignees View on GitHub
area/core area/tinkerpop kind/enhancement
Dominant language
Java
Stars
5.8k
Forks
1.2k
Avg merge
13h 53m
Merged PRs (30d)
6

Description

Some VertexPrograms store halted traversers in memory so that other vertex programs / iterations of the same program can continue with these traversers. This requires serialization of the traversed elements which can be JanusGraph internal elements like `CacheVertex`.

If for example two vertex programs are executed after another for the same traversal where vertices are traversed in between, then the following exception is thrown:

```java
org.janusgraph.core.JanusGraphException: Could not commit transaction due to exception during persistence

at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1393)
at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction.compute(JanusGraphBlueprintsTransaction.java:75)
at org.apache.tinkerpop.gremlin.process.computer.Computer.apply(Computer.java:111)
at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:75)
[...]
Caused by: org.apache.commons.lang3.SerializationException: java.io.NotSerializableException: org.janusgraph.graphdb.vertices.CacheVertex
at org.apache.commons.lang3.SerializationUtils.serialize(SerializationUtils.java:157)
at org.apache.commons.lang3.SerializationUtils.serialize(SerializationUtils.java:179)
at org.janusgraph.graphdb.database.serialize.attribute.SerializableSerializer.write(SerializableSerializer.java:44)
at org.janusgraph.graphdb.database.serialize.attribute.SerializableSerializer.write(SerializableSerializer.java:31)
at org.janusgraph.graphdb.database.serialize.StandardSerializer$StandardDataOutput.writeObjectNotNullInternal(StandardSerializer.java:336)
at org.janusgraph.graphdb.database.serialize.StandardSerializer$StandardDataOutput.writeClassAndObject(StandardSerializer.java:347)
at org.janusgraph.graphdb.database.EdgeSerializer.writePropertyValue(EdgeSerializer.java:371)
at org.janusgraph.graphdb.database.EdgeSerializer.writePropertyValue(EdgeSerializer.java:365)
at org.janusgraph.graphdb.database.EdgeSerializer.writeRelation(EdgeSerializer.java:288)
at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:606)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:726)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382)
[...]
```

I only managed to reproduce this problem with two vertex programs executed after another and even only with the newly introduced [`ConnectedComponentVertexProgram`](http://tinkerpop.apache.org/docs/current/reference/#connectedcomponentvertexprogram). It can be reproduced like this:

```java
g.V().dedup().connectedComponent().toList();
```

While the `ConnectedComponentVertexProgram` is only available starting with TinkerPop 3.4.0 which we don't support yet, I'm not sure if this problem can't also be encountered with different vertex programs.

Making serialization of JanusGraph's types like `CacheVertex` possible is unfortunately not a trivial task as they have a `StandardJanusGraphTx` member that can't be easily serialized. So, we might need to find some other workaround.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with g.V().dedup().connectedComponent().toList(), then trace the serialization path through SerializableSerializer, StandardSerializer, and EdgeSerializer. Determine a workable way to persist halted traversers containing CacheVertex without serializing its StandardJanusGraphTx; done means the traversal no longer raises NotSerializableException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.