apache / apache/arrow

[R] read_ipc_stream fails to unify nested Enum fields inside a Struct array across record batches

Open
#50,514 1 comment 0 reactions 1 assignee Claimed by @thisisnic View on GitHub
Component: R Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

When reading an Arrow IPC stream that contains a nested Struct array named resources, the field "type"is an Enum in the source API schema (described here: https://md.umwelt.info/swagger-ui/ at the bottom in Dataset/resources/type).

The R arrow package tries to cast this as a factor, but this produces an invalid dataframe.
If the column is transferred to character it works fine.

Minimal Reproducible Example
```
library(arrow)
library(httr2)
library(tidyr)
library(dplyr)

req <- httr2::request("https://md.umwelt.info/search/all?format=arrow_ipc") |>
httr2::req_url_query(
query = "type:'/Daten und Messstellen/Wasser/Flüsse' AND measuring_station:true",
language = "de"
)

resp <- httr2::req_perform(req)
raw_bytes <- httr2::resp_body_raw(resp)

df <- as.data.frame(arrow::read_ipc_stream(raw_bytes)) |>
unnest("resources", names_sep = "_")

glimpse(df) # yields: Error in as.character.factor(x) : malformed factor
glimpse(df[,21]) # resources_types is the field that causes the error
```

Environment

OS: Ubuntu 26.04 LTS

R Version: 4.5.2

arrow R Package Version: 25.0.0.9000

### Component(s)

R

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.