[R] ChunkedArray$create assumes all chunks are the same type
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
It detects the type from the first chunk and uses it for all chunks. Normally this works ok, but it can lead to unexpected behavior, such as:
```r
data <- list(1:10, NaN)
x <- chunked_array(!!!data)
```
returns:
`Error: Invalid: Value is too large to fit in C integer type`
There are a few things that might fix/change this:
- improved error message
- chunked arrays not assuming the first chunk's types can be cast safely to all others
Note that in this case, specifying the type to int64() does "work" with an overflowed NaN value (-9223372036854775808)
```r
data <- list(1:10, NaN)
x <- chunked_array(!!!data, type = int64())
```
**Reporter**: [Jonathan Keane](https://issues.apache.org/jira/browse/ARROW-11211) / @jonkeane
**Note**: *This issue was originally created as [ARROW-11211](https://issues.apache.org/jira/browse/ARROW-11211). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start at the R entry points ChunkedArray$create and chunked_array, then reproduce the reported list(1:10, NaN) example and compare it with the explicit int64() case. Determine the intended behavior for mixed chunk types and define completion as consistent handling with an informative result or error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100