apache / apache/arrow

[Python] Nullability not maintained when casting a column

Open
#37,219 5 comments 0 reactions 0 assignees View on GitHub
Component: Python good-second-issue Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Given the following Parquet file:
```
➜ arrow git:(arrow-14705) ✗ parquet-tools inspect ~/Desktop/int.parquet

############ file meta data ############
created_by: parquet-cpp-arrow version 12.0.0
num_columns: 1
num_rows: 3
num_row_groups: 1
format_version: 2.6
serialized_size: 798

############ Columns ############
id

############ Column(id) ############
name: id
path: id
max_definition_level: 0
max_repetition_level: 0
physical_type: INT32
logical_type: None
converted_type (legacy): NONE
compression: SNAPPY (space_saved: -5%)
```

When I widen a type, I see that the not-null is dropped from the schema:

![image](https://github.com/apache/arrow/assets/1134248/a06172d3-5626-416f-b5c0-a8a536179643)

I think this should be maintained.

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.