apache / apache/fluss-rust

Implement ArrowWriter pooling

Open
#444 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
58
Forks
50
PR merge metrics
No merged PRs in 30d

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss-rust/issues) and found nothing similar.

### Description

Java pools ArrowWriter instances per table/schema via ArrowWriterPool. Reused writers keep their allocated vectors and seed the fullness threshold from the previous batch (recordsCount / 2), avoiding reallocation and the first-record size check on every new batch. We currently create a fresh MemoryLogRecordsArrowBuilder per batch.

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.