apache / apache/arrow

[C++] RecordBatch::ValidateFull fails on nested StructArray

Open
#31,369 4 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

ValidateFull appears to discard the outermost field of nested schema. The following example passes:

```bash

diff --git a/cpp/src/arrow/array/array_struct_test.cc b/cpp/src/arrow/array/array_struct_test.cc
index 318c83860..6a8896ca9 100644
--- a/cpp/src/arrow/array/array_struct_test.cc
+++ b/cpp/src/arrow/array/array_struct_test.cc
@@ -15,6 +15,8 @@
// specific language governing permissions and limitations
// under the License.

+#include
+
#include

#include
@@ -696,4 +698,20 @@ TEST(TestFieldRef, GetChildren) {
AssertArraysEqual(*a, *expected_a);
}

+TEST(TestFieldRef, TestValidateFullRecordBatch) {
+ auto struct_array =
+ ArrayFromJSON(struct_({field("a", struct_({field("b", float64())}))}), R"([
+ {"a": {"b": 6.125}},
+ {"a": {"b": 0.0}},
+ {"a": {"b": -1}}
+ ])");
+
+ auto schema1 = arrow::schema({field("x", struct_({field("a", struct_({field("b", float64())}))}))});
+ auto schema2 = arrow::schema({field("a", struct_({field("b", float64())}))});
+ auto record_batch1 = arrow::RecordBatch::Make(schema1, 3, {struct_array});
+ auto record_batch2 = arrow::RecordBatch::Make(schema2, 3, {struct_array});
+ ASSERT_OK(record_batch1->ValidateFull());
+ ASSERT_NOT_OK(record_batch2->ValidateFull());
+}
+
```

Is this expected behaviour?

**Reporter**: [Rok Mihevc](https://issues.apache.org/jira/browse/ARROW-15942) / @rok
**Watchers**: [Rok Mihevc](https://issues.apache.org/jira/browse/ARROW-15942) / @rok

**Note**: *This issue was originally created as [ARROW-15942](https://issues.apache.org/jira/browse/ARROW-15942). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start with cpp/src/arrow/array/array_struct_test.cc and the TestFieldRef::TestValidateFullRecordBatch reproduction. Run the relevant C++ test and inspect RecordBatch::ValidateFull for the two nested schemas. Done means the expected validation behavior is established and covered by a passing regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.