apache / apache/sedona-db

`get_array_data_memory_size` does not return correct results for listarray slices

Open
#519 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
503
Forks
61
Avg merge
2d 4h
Merged PRs (30d)
90

Description

arrow-rs propagates the slice for a StructArray on slice(): https://docs.rs/arrow-array/57.2.0/src/arrow_array/array/struct_array.rs.html#337

It doesn't do this for list arrays: https://docs.rs/arrow-array/57.2.0/src/arrow_array/array/list_array.rs.html#396 ...and possibly not for some other types like dictionaries, list views, or run-end-encoded

See this comment https://github.com/apache/sedona-db/pull/515#discussion_r2694835218 and the failing test case: https://github.com/apache/sedona-db/pull/515/files#diff-5fdc254a5ea41059acecb8588e449aae6b354d4886deecbe3b2ffec3628c4b06R257-R290

Contributor guide

Open the contributing guide

Research direction

Start with get_array_data_memory_size and the failing test case linked from pull request #515. Compare how arrow-rs propagates slices for StructArray and list arrays, then check the mentioned dictionary, list-view, and run-end-encoded cases. Done means sliced list arrays return correct memory sizes and the failing test passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.