[Python][Parquet] Allow `write_batch` to directly write batch
Open
Component: Parquet
Component: Python
Type: enhancement
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
Currently, Python parquet's `ParquetWriter.write_batch` will fallback to call the `write_table`, `write_table` will call `NewRowGroup`, after write, it will finish the RowGroup, which will causing lots of RowGroups if batch is small.
Nowadays, `WriteRecordBatch` is supported in C++ ParquetWriter, so, it's possible that Python use `WriteRecordBatch` rather than `WriteTable`.
### Component(s)
Parquet, Python
Contributor guide
Assessment
This issue has not been assessed yet.