Unable to serialize large R*Tree using FlatBuffer
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 218
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
I was trying to Serialize a large R*Tree using the APIs in this Library, but it failed with error :
Exception in thread "main" java.lang.AssertionError: FlatBuffers: cannot grow buffer beyond 2 gigabytes.
at com.google.flatbuffers.FlatBufferBuilder.growByteBuffer(FlatBufferBuilder.java:127)
at com.google.flatbuffers.FlatBufferBuilder.prep(FlatBufferBuilder.java:173)
at com.google.flatbuffers.FlatBufferBuilder.addOffset(FlatBufferBuilder.java:291)
at com.google.flatbuffers.FlatBufferBuilder.addOffset(FlatBufferBuilder.java:562)
at com.github.davidmoten.rtree.fbs.generated.Entry_.addObject(Entry_.java:33)
at com.github.davidmoten.rtree.fbs.generated.Entry_.createEntry_(Entry_.java:26)
at com.github.davidmoten.rtree.fbs.FlatBuffersHelper.addEntries(FlatBuffersHelper.java:77)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:96)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.addNode(SerializerFlatBuffers.java:102)
at com.github.davidmoten.rtree.fbs.SerializerFlatBuffers.write(SerializerFlatBuffers.java:73)
Seems like FlatBuffers can't serialize large structures with size more than 2GB? Am I mistaken here?
Does this implementation support any other serialization apart from FlatBuffer?
I saw few test cases for Kryo Serialization but don't think if the current structure supports this.
Please guide.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.