structs: Support casting nullable fields to non-nullable if there are no null values
- Dominant language
- Assembly
- Stars
- 404
- Forks
- 145
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 87
Description
This solves https://github.com/apache/arrow/issues/33592
This is a sister PR to https://github.com/apache/arrow/pull/43782, I was told to open a PR here to make this change on the Go side.
I need a few pointers though if you want me to do the implementation. Could you give me a high-level summary of what needs to happen? I did an initial exploration and had a few notes/questions:
- I went looking for an equivalent nullable to non-nullable cast test for non-struct arrays, but couldn't find any. Is this possibly not tested? Or do I misunderstand the arrow spec, and all datatypes EXCEPT struct fields are nullable, so there is no eg non-nullable uint16?
- More philosophically, is this behavior actually in the arrow spec? Or is the spec just about the in-memory format, and implementations are free to define their compute/casting behavior as desired? It seems funny/non-ideal to me that it is possible for the go implementation of this casting to drift away from the c++ implementation. I would think it would be better if there was some test that verified they both had the same behavior?
- Same thing for the actual implementation. I wanted to use these as examples to work from.
- Note for myself/fixer: [here is the struct test](https://github.com/apache/arrow-go/blob/12482ca3eb1cd4da21c45325e174f1dac871281b/arrow/compute/cast_test.go#L2639) that will need to get updated
Contributor guide
Assessment
This issue has not been assessed yet.