apache / apache/arrow

[Python][Interchange] from_dataframe drops the categorical ordered flag

Open
#51,046 0 comments 0 reactions 1 assignee Claimed by @Kayvan-Zahiri View on GitHub
Component: Python
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

`pyarrow.interchange.from_dataframe` builds the dictionary array without the `is_ordered` flag the producer reports, so an ordered categorical column comes back unordered. A pyarrow table whose column is `dictionary(int32, string, ordered=True)` does not survive its own round trip, and an ordered pandas `Categorical` loses its ordering on the way in.

```python
import pyarrow as pa
from pyarrow.interchange import from_dataframe

t = pa.table({"c": pa.array(["a", "b", "a"]).dictionary_encode().cast(
pa.dictionary(pa.int32(), pa.string(), ordered=True))})
print(t.schema.field("c").type.ordered) # True
print(from_dataframe(t).schema.field("c").type.ordered) # False
```

The interchange protocol exposes the flag: a categorical column's `describe_categorical` returns `is_ordered`, and the pandas consumer passes it through. The pyarrow consumer reads the mapping but drops `is_ordered`.

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.