`get_array_data_memory_size` does not return correct results for listarray slices
- Dominant language
- Rust
- Stars
- 503
- Forks
- 61
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 90
Description
arrow-rs propagates the slice for a StructArray on slice(): https://docs.rs/arrow-array/57.2.0/src/arrow_array/array/struct_array.rs.html#337
It doesn't do this for list arrays: https://docs.rs/arrow-array/57.2.0/src/arrow_array/array/list_array.rs.html#396 ...and possibly not for some other types like dictionaries, list views, or run-end-encoded
See this comment https://github.com/apache/sedona-db/pull/515#discussion_r2694835218 and the failing test case: https://github.com/apache/sedona-db/pull/515/files#diff-5fdc254a5ea41059acecb8588e449aae6b354d4886deecbe3b2ffec3628c4b06R257-R290
Contributor guide
Research direction
Start with get_array_data_memory_size and the failing test case linked from pull request #515. Compare how arrow-rs propagates slices for StructArray and list arrays, then check the mentioned dictionary, list-view, and run-end-encoded cases. Done means sliced list arrays return correct memory sizes and the failing test passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100