[C++] write_csv tries to write binary columns as UTF-8 and fails if they're invalid
Open
Component: C++
Component: Python
Type: bug
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
Tested on pyarrow-16.1.0 but also confirmed on the C++ API.
Minimal reproducer:
```
>>> import pyarrow as pa
>>> import pyarrow.csv
>>> import sys
>>> pa.csv.write_csv(pa.Table.from_arrays([pa.array([b'\xa2'])], names=["data"]), sys.stdout.buffer)
ArrowInvalid: Invalid UTF8 payload
```
It should probably write out an escaped string instead.
### Component(s)
C++, Python
Contributor guide
Assessment
This issue has not been assessed yet.