apache / apache/arrow-rs

Improve parquet ArrayReaderBuilder tests

Open
#9,921 2 comments 0 reactions 0 assignees View on GitHub
good first issue
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

As an API suggestion (maybe for another PR, not required):

I would personally find this much easier to understand if the fields were named so I didn't have to refer to `make_int32_page_reader` to figure out what`&[0, 1, 2, 3, 1]` and `&[0, 1, 1, 1, 1]` meant.

Perhaps we could do this with an builder in the test. Something like this perhaps:

```rust
let array_reader_1 = TestArrayReaderBuilder::new()
.with_non_null_i32_values(&[4]) // <-- this is especially non obvous if you are used to normal Arrow arrays
.with_def_levels(&[0, 1, 2, 3, 1])
.with_rep_levels(&[0, 1, 1, 1, 1])
.with_max_def_level(3)
.with_max_rep_level(1)
.build()
```

_Originally posted by @alamb in https://github.com/apache/arrow-rs/pull/9847#discussion_r3173223954_

Contributor guide

Open the contributing guide

Research direction

Start with the ArrayReaderBuilder tests and the make_int32_page_reader helper referenced in the issue and PR discussion. Extract the test setup into a builder whose fields make the non-null values, definition levels, repetition levels, and maximum levels explicit; done means the tests are easier to understand without decoding the numeric arrays from the helper.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.