[Java] enable zero-copy in `serialize(OutputStream)`
- Dominant language
- Java
- Stars
- 4.5k
- Forks
- 443
- Avg merge
- 6h 3m
- Merged PRs (30d)
- 74
Description
## Is your feature request related to a problem? Please describe.
For large graph, current `serialize(OutputStream)` will serialize data into a binary heap buffer, then write it into `OutputStream`, which is not really streaming.
Large graph will have many binary buffers inside, serializing without streaming will introduce an extra copy, and can't overlap network. If we enable fury zero-copy support, all binary buffer will be handled separately without writting it into the heap buffer, which will reduce memory copy, and serialization time. And then we can send in-band buffers and all out-of-band buffers chunk by chunk to speed up the transfer.
## Describe the solution you'd like
Make `serialize(OutputStream)` use bufferCallback to capture binary buffer:
```java
List buffers = new ArrayList<>();
serialize(outputStream, obj, o -> {
buffers.add(o);
return false;
});
buffers.forEach(bufferObject -> {
int size = bufferObject.totalBytes();
byte[] sizeBytes = toBytes(size);
outputStream.write(sizeBytes);
bufferObject.writeTo(outputStream);
});
```
Potential disadvantages:
If all binary buffers in object graph are small objects, then the serialization will be slightly slower.
Contributor guide
Assessment
This issue has not been assessed yet.