JanusGraph / JanusGraph/janusgraph
Enable serialization of JanusGraph element types
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
Some VertexPrograms store halted traversers in memory so that other vertex programs / iterations of the same program can continue with these traversers. This requires serialization of the traversed elements which can be JanusGraph internal elements like `CacheVertex`.
If for example two vertex programs are executed after another for the same traversal where vertices are traversed in between, then the following exception is thrown:
```java
org.janusgraph.core.JanusGraphException: Could not commit transaction due to exception during persistence
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1393)
at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction.compute(JanusGraphBlueprintsTransaction.java:75)
at org.apache.tinkerpop.gremlin.process.computer.Computer.apply(Computer.java:111)
at org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:75)
[...]
Caused by: org.apache.commons.lang3.SerializationException: java.io.NotSerializableException: org.janusgraph.graphdb.vertices.CacheVertex
at org.apache.commons.lang3.SerializationUtils.serialize(SerializationUtils.java:157)
at org.apache.commons.lang3.SerializationUtils.serialize(SerializationUtils.java:179)
at org.janusgraph.graphdb.database.serialize.attribute.SerializableSerializer.write(SerializableSerializer.java:44)
at org.janusgraph.graphdb.database.serialize.attribute.SerializableSerializer.write(SerializableSerializer.java:31)
at org.janusgraph.graphdb.database.serialize.StandardSerializer$StandardDataOutput.writeObjectNotNullInternal(StandardSerializer.java:336)
at org.janusgraph.graphdb.database.serialize.StandardSerializer$StandardDataOutput.writeClassAndObject(StandardSerializer.java:347)
at org.janusgraph.graphdb.database.EdgeSerializer.writePropertyValue(EdgeSerializer.java:371)
at org.janusgraph.graphdb.database.EdgeSerializer.writePropertyValue(EdgeSerializer.java:365)
at org.janusgraph.graphdb.database.EdgeSerializer.writeRelation(EdgeSerializer.java:288)
at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:606)
at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:726)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1382)
[...]
```
I only managed to reproduce this problem with two vertex programs executed after another and even only with the newly introduced [`ConnectedComponentVertexProgram`](http://tinkerpop.apache.org/docs/current/reference/#connectedcomponentvertexprogram). It can be reproduced like this:
```java
g.V().dedup().connectedComponent().toList();
```
While the `ConnectedComponentVertexProgram` is only available starting with TinkerPop 3.4.0 which we don't support yet, I'm not sure if this problem can't also be encountered with different vertex programs.
Making serialization of JanusGraph's types like `CacheVertex` possible is unfortunately not a trivial task as they have a `StandardJanusGraphTx` member that can't be easily serialized. So, we might need to find some other workaround.
Contributor guide
Research direction
Start by reproducing the failure with g.V().dedup().connectedComponent().toList(), then trace the serialization path through SerializableSerializer, StandardSerializer, and EdgeSerializer. Determine a workable way to persist halted traversers containing CacheVertex without serializing its StandardJanusGraphTx; done means the traversal no longer raises NotSerializableException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100