apache / apache/uniffle

[Improvement] Avoid copying direct memory when flushing data.

Open
#2,092 3 comments 0 reactions 1 assignee Claimed by @zhengchenyu View on GitHub
Dominant language
Java
Stars
454
Forks
172
Avg merge
5d 17h
Merged PRs (30d)
5

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.

### What would you like to be improved?

When flush data to local disk, will copy data. See code below:

https://github.com/apache/incubator-uniffle/blob/0f9d9bcb5648c95051c9f8c4831943f126d463b9/storage/src/main/java/org/apache/uniffle/storage/handler/impl/LocalFileWriteHandler.java#L106

In fact, we can direct write ByteBuffer to file channel. For netty mode, the ByteBuffer is DirectByteBuffer, will avoid copy the memory.

### How should we improve?

_No response_

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.