[Improvement] Use ByteString#asReadOnlyByteBuffer to reduce the memory allocation and GC pressure
- Dominant language
- Java
- Stars
- 454
- Forks
- 172
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 5
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
In our production environment, the shuffle server is always stopped due to the full GC. After increasing more memory, it still occurs, that makes me confused. So I suspect some abusing memory exists in uniffle.
After digging into the Grpc service api of `sendShuffleData`, I found something is abnormal. Please see this part code.
https://github.com/apache/incubator-uniffle/blob/1fbdfe576f0a164f173cbff9ad83dccccb909ee2/server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java#L751-L769
The byte[] will always be created in the invoking of `block.getData().toByteArray()`, which will cause GC if receiving data speed is high. And this has been proved in our dashboard
Besides, the byte[] will be created when reading shuffle data in memory, which may be not the main factor to trigger this. Anyway, it also should be fixed.
### How should we improve?
We could use the `block.getData().asReadOnlyByteBuffer()` to replace `toByteArray` to avoid unnecessary memory allocation.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Contributor guide
Assessment
This issue has not been assessed yet.