[Python] Convert maps to dicts without per-element Scalars in to_pylist(maps_as_pydicts=...)
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
Follow-up of #50326. GH-50327 makes `Array.to_pylist()` convert without per-element Scalars, but the `maps_as_pydicts` option still routes to the Scalar-based path: every map row allocates a `MapScalar`, converts its keys via per-element `as_py`, and builds the dict in Python.
Converting maps to dicts is the natural consumption pattern for engines whose map values are Python dicts (e.g. Spark's Arrow-serialized Python UDFs currently receive association lists from `to_pylist()` and rebuild a dict per row in pure Python — one of the dominant remaining costs in that path).
Proposal: thread `maps_as_pydicts` through the scalar-free `_getitem_py` mechanism introduced in GH-50327 so `MapArray` builds the dict directly from the flattened keys/items children:
- default (`None`) semantics unchanged (association lists);
- `'lossy'`/`'strict'` build the dict in one pass; when the dict size shows duplicate keys, the row is redone with the careful per-key loop so the warning ('lossy') and `KeyError` ('strict') semantics stay exactly identical to `MapScalar.as_py`;
- invalid values still raise the same `ValueError` when a map value is converted;
- the option propagates through nested types (list/struct/map values) as before, and unspecialized types keep the exact Scalar fallback.
### Component(s)
Python
Contributor guide
Assessment
This issue has not been assessed yet.