Arrow IPC integration test "generated_map_non_canonical" failure
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 167
Description
**Describe the bug**
There is some problem reading the Arrow Map files from the IPC format
in the reader tests: https://github.com/apache/arrow-rs/blob/81abc1a942cd13a92231f4a828077ad60fdabe36/arrow-integration-testing/tests/ipc_reader.rs#L120-L122
and the writer tests: https://github.com/apache/arrow-rs/blob/81abc1a942cd13a92231f4a828077ad60fdabe36/arrow-integration-testing/tests/ipc_writer.rs#L73-L74
**To Reproduce**
Uncomment one the lines referenced above and the test will fail like the following (looks like the names of the fields are different -- "entries" vs "some_entries")
```
Verifying /Users/alamb/Software/arrow-rs2/arrow/../testing/data/arrow-ipc-stream/integration/1.0.0-littleendian/generated_map_non_canonical.stream
verifying content
thread 'read_1_0_0_littleendian' panicked at 'assertion failed: `(left == right)`
left: `Field { name: "map_other_names", data_type: Map(Field { name: "some_entries", data_type: Struct([Field { name: "some_key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "some_value", data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }`,
right: `Field { name: "map_other_names", data_type: Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }`: Arrow fields not the same', arrow-integration-test/src/lib.rs:253:17
stack backtrace:
0: rust_begin_unwind
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/std/src/panicking.rs:575:5
1: core::panicking::panic_fmt
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:65:14
2: core::panicking::assert_failed_inner
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:240:23
3: core::panicking::assert_failed
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/panicking.rs:203:5
4: arrow_integration_test::ArrowJsonField::equals_field
at /Users/alamb/Software/arrow-rs2/arrow-integration-test/src/lib.rs:253:17
5: arrow_integration_test::ArrowJsonSchema::equals_schema
at /Users/alamb/Software/arrow-rs2/arrow-integration-test/src/lib.rs:217:17
6: arrow_integration_test::ArrowJson::equals_reader
at /Users/alamb/Software/arrow-rs2/arrow-integration-test/src/lib.rs:166:13
7: ipc_reader::verify_arrow_stream
at ./tests/ipc_reader.rs:218:17
8: ipc_reader::read_1_0_0_littleendian::{{closure}}
at ./tests/ipc_reader.rs:140:9
9: as core::iter::traits::iterator::Iterator>::for_each
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/slice/iter/macros.rs:211:21
10: ipc_reader::read_1_0_0_littleendian
at ./tests/ipc_reader.rs:138:5
11: ipc_reader::read_1_0_0_littleendian::{{closure}}
at ./tests/ipc_reader.rs:109:1
12: core::ops::function::FnOnce::call_once
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/ops/function.rs:251:5
13: core::ops::function::FnOnce::call_once
at /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/ops/function.rs:251:5
```
**Expected behavior**
**Additional context**
Found on https://github.com/apache/arrow-rs/pull/3427
Contributor guide
Research direction
Start by uncommenting the referenced cases in arrow-integration-testing/tests/ipc_reader.rs and ipc_writer.rs, then run the failing integration tests. Read arrow-integration-test/src/lib.rs around ArrowJsonField::equals_field and compare the generated_map_non_canonical schema fields. Done means the reader and writer integration tests pass for this Map file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100