apache / apache/arrow-go

RecordBuilder should confirm non-nullable fields do not contain null rows when building Record

Open
#372 1 comment 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Assembly
Stars
404
Forks
145
Avg merge
2d 4h
Merged PRs (30d)
87

Description

### Describe the bug, including details regarding any error messages, version, and platform.

I am able to construct an arrow record, where a field should be non-nullable, but it contains null rows. I'm wondering if maybe `RecordBuilder.NewRecord()` should confirm that the fields's arrays do not contain nulls if the field is specified as `Nullable: false` in the schema?

version = v18.2.0
platform = arm mac

```go
package main

import (
"log"

"github.com/apache/arrow-go/v18/arrow"
"github.com/apache/arrow-go/v18/arrow/array"
"github.com/apache/arrow-go/v18/arrow/memory"
)

func main() {
allocator := memory.NewGoAllocator()
schema := arrow.NewSchema([]arrow.Field{
// should be not null:
{Name: "a", Type: arrow.PrimitiveTypes.Int32, Nullable: false},
}, nil)
recordBuilder := array.NewRecordBuilder(allocator, schema)

recordBuilder.Field(0).(*array.Int32Builder).Append(1)
recordBuilder.Field(0).(*array.Int32Builder).Append(2)
recordBuilder.Field(0).AppendNull()
recordBuilder.Field(0).(*array.Int32Builder).Append(4)

record := recordBuilder.NewRecord() // this works
defer record.Release()

log.Printf("record: %v", record)
/* prints
2025/05/08 20:32:47 record: record:
schema:
fields: 1
- a: type=int32
rows: 4
col[0][a]: [1 2 (null) 4]
*/
}
```

This is not allowed in the Rust implementation:
```rs
use std::sync::Arc;

use arrow_array::{Int32Array, RecordBatch};
use arrow_schema::{DataType, Field, Schema};

fn main() {
let schema = Arc::new(Schema::new(vec![Field::new("a", DataType::Int32, false)]));
let mut ids = Int32Array:: builder(4);
ids.append_value(1);
ids.append_value(2);
ids.append_null();
ids.append_value(4);
let ids = ids.finish();
// this panics:
// called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Column 'a' is declared as non-nullable but contains null values")
let batch = RecordBatch::try_new(schema.clone(), vec![Arc::new(ids)]).unwrap();
println!("batch: {:?}", batch);
}
```

### Component(s)

Other

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.