[Python] write_to_dataset(max_row_group_size)
Open
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
I converted from using `pyarrow.dataset.write_dataset` to `pyarrow.parquet.write_to_dataset` and noticed that this section of code is overwriting my max_row_group_size param to None. It seems it forces the user to pass in chunksize or row_group_size. I think an additional kwargs.pop would be necessary here.
https://github.com/apache/arrow/blob/71a7b5531fd6ef96ce52d04491656031eb023c09/python/pyarrow/parquet/core.py#L2194-L2196
Contributor guide
Assessment
This issue has not been assessed yet.