JanusGraph / JanusGraph/janusgraph
Kryo registrator options for Spark
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
This came up during a [documentation review](https://github.com/JanusGraph/janusgraph/pull/1177#issuecomment-408504616):
https://github.com/JanusGraph/janusgraph/blob/0.2/janusgraph-hadoop-parent/janusgraph-hadoop-core/src/test/resources/cassandra-read.properties#L30
Should we use:
> spark.kryo.registrator=org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator
Or:
> spark.kryo.registrator=org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoRegistrator
`JanusGraphKryoRegistrator` only registers a serializer for `Geoshape`, while the TinkerPop `GryoRegistrator` handles many different classes. `JanusGraphKryoRegistrator` does not extend `GryoRegistrator`.
We need some clarity on how to configure the Kryo registrator to cover all required TinkerPop and JanusGraph classes. Also will need updates on the documentation and the properties files in the distribution.
Contributor guide
Research direction
Start with janusgraph-hadoop-parent/janusgraph-hadoop-core/src/test/resources/cassandra-read.properties at the referenced Spark setting, then review the linked documentation review. Determine which registrator covers the required TinkerPop and JanusGraph classes, and identify the distribution properties files and documentation that need matching updates. Done means the configuration choice is resolved and those references are consistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- databases, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100