[Go] Estimate physical size of data accumulated in array.Builder
- Dominant language
- Assembly
- Stars
- 404
- Forks
- 145
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 87
Description
### Describe the usage question you have. Please include as many useful details as possible.
I'm working on a streaming service that accumulates blocks of Arrow data, dumps them into Arrow Streaming IPC format and then writes them into network socket. The data is accumulated in `[]array.Builder`, which converts into `[]arrow.Array`, then into a single `arrow.Record` and finally into `bytes.Buffer`.
The key part of this algorithm is the separation of a new block: we need to know when the previous block is large enough (but not too large - because it could exceed message size limits in this case) to let us stop data accumulation in `[]array.Builder`, serialize existing data into `bytes.Buffer` and send over the network.
So is it possible to estimate the number of bytes accumulated physically in `array.Builder`? Or may be it even possible to forecast the size of a future `bytes.Buffer` containing serialized data from `[]array.Builder`?
### Component(s)
Go
Contributor guide
Assessment
This issue has not been assessed yet.