apache / apache/arrow

[R] "Error: Filter expression not supported for Arrow Datasets" using "date" expression rigth hand side of a filter

Open
#39,688 2 comments 0 reactions 1 assignee Claimed by @thisisnic View on GitHub
Component: Parquet Component: R Priority: Critical Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Hello,

Here is a repex of a bug occuring when I try to filter a dataset like "filter(Date == date).
Maybe the expression "date" has a special interpretation ?

Thx,

```
library(arrow)
library(dplyr)
library(lubridate)

# Create a simple df
set.seed(123)
Date <- rep(c("2022-01-01", "2022-01-02"), each = 5)
Volume <- sample(100:200, 10, replace = TRUE)

df <- tibble(
Date = Date,
Volume = Volume
)

# save as partitioned parquet
temp_dir <- tempdir()
write_dataset(df, temp_dir, partitioning = "Date")

# Lazy import
reimported_data <- open_dataset(temp_dir)

# Bug
date = "2022-01-02"
reimported_data |> filter(Date == date)

# Possible workaround
my_date = "2022-01-02"
reimported_data |> filter(Date == my_date)

```

### Component(s)

Parquet, R

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.