apache / apache/arrow-dotnet

The construction of arrays should be more flexible

Open
#97 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
39
Forks
30
Avg merge
1d 9h
Merged PRs (30d)
16

Description

Today, arrays can only be constructed using a type-specific `Builder`. Builders start with some default initial size and grow either incrementally or by setting a new capacity. Once construction is complete, an immutable `IArrowArray` is produced. This mode of operation is safe but inflexible and potentially inefficient.

Some ideas for improving this are to
- Allow an initial size to be specified for builders
- Support (fixed-size) read-write arrays -- at least for fixed-width types -- in addition to read-only arrays
- Allow ownership of memory to pass from the builder to the array upon construction so as not to require an additional allocation.
- Allow native memory to be allocated without clearing it first
- Allow a null value to be set directly in the builder, just as a non-null value can be
- More flexible builders for nested types
- Validating that the nullability of the type matches the values in the array

Inspired by:
[https://github.com/apache/arrow/issues/21093](https://github.com/apache/arrow/issues/21093)
[https://github.com/apache/arrow/issues/21386](https://github.com/apache/arrow/issues/21386)
[https://github.com/apache/arrow/issues/21746](https://github.com/apache/arrow/issues/21746)
[https://github.com/apache/arrow/issues/22309](https://github.com/apache/arrow/issues/22309)
[https://github.com/apache/arrow/issues/25449](https://github.com/apache/arrow/issues/25449)
[https://github.com/apache/arrow/issues/30567](https://github.com/apache/arrow/issues/30567)
[https://github.com/apache/arrow/issues/35285](https://github.com/apache/arrow/issues/35285)
[https://github.com/apache/arrow/issues/36133](https://github.com/apache/arrow/issues/36133)
[https://github.com/apache/arrow/issues/40188](https://github.com/apache/arrow/issues/40188)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.