apache / apache/arrow

[Python] Converting Table to pandas raises NotImplementedError (when table previously saved as partitioned parquet dataset)

Open
#20,400 5 comments 0 reactions 1 assignee Claimed by @jorisvandenbossche View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

When converting a table in which one of the column's type is of DictionaryType (values=int32, indices=int32, ordered=0) the conversion to pandas DataFrame fails with:

NotImplementedError: dictionary

The dictionary has this conversion not implmented yet.

This DictionaryType is used as type when using one of the columns (Int64) as one of the parquet's dataset partition columns.

**Environment**: Docker container, based on continuumio/anaconda3
Python 3.9.12
PyArrow 9.0.0
**Reporter**: [Roberto Lobo](https://issues.apache.org/jira/browse/ARROW-17636)
**Assignee**: [Joris Van den Bossche](https://issues.apache.org/jira/browse/ARROW-17636) / @jorisvandenbossche
#### Original Issue Attachments:
- [bug.py](https://issues.apache.org/jira/secure/attachment/13049733/bug.py)

**Note**: *This issue was originally created as [ARROW-17636](https://issues.apache.org/jira/browse/ARROW-17636). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.