apache / apache/parquet-java

Filtering dictionary data types using dplyr in Arrow package

Open
#2,562 0 comments 0 reactions 0 assignees View on GitHub
Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

Is there any way where we can filter the dictionary data types using dplyr. Below is the example for illustration 

test %>% dplyr::filter(cols %in% c("A","B","C") %>collect(). It gives me following error

Error in dataset___Scanner__ToTable(self) :   NotImplemented: Function is_in has no kernel matching input types (array[dictionary])

Here cols is dictionary type value

**Reporter**: [Soumya Ranjan Prusty](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=prustys)

**Note**: *This issue was originally created as [PARQUET-1962](https://issues.apache.org/jira/browse/PARQUET-1962). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.