Support large arrays
- Dominant language
- C#
- Stars
- 39
- Forks
- 30
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 16
Description
### Describe the enhancement requested
It's hard for the C# Arrow library to support large arrays. Commonly-used types like `Array`, `Span` and `Memory` use a 32-bit signed integer for the length, meaning that they can't have more than ~2 million elements in them. The `ArrowBuffer` structure that is the foundation of the core `ArrayData` class exposes these types directly. This may make it impossible to support large arrays in a strictly backwards-compatible fashion.
Some previously-opened items related to this feature are:
[https://github.com/apache/arrow/issues/23776](https://github.com/apache/arrow/issues/23776)
[https://github.com/apache/arrow/issues/38086](https://github.com/apache/arrow/issues/38086)
[https://github.com/apache/arrow/pull/38619](https://github.com/apache/arrow/pull/38619)
[https://github.com/apache/arrow/issues/45030](https://github.com/apache/arrow/issues/45030)
Contributor guide
Assessment
This issue has not been assessed yet.