apache / apache/arrow

[R] Update `write_csv_arrow()` implementation to match `readr::write_csv()`

Open
#36,864 9 comments 0 reactions 0 assignees View on GitHub
Component: R Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

The implementation of `arrow::read_csv_arrow()` intentionally mirrors `readr::read_csv`. However, the implementation of `arrow::write_csv_arrow()` doesn't closely mirror `readr::write_csv()`.

Now that we have additional convenience functions for `write_dataset()` being implemented in #36436, this presents a problem in that the `open_dataset()` convenience wrapper functions use the parameters from the `read_*` functions to provide a common interface which also reflect the readr interface. However, at present, if we implement the `write_*_dataset()` to mirror the `write_csv_arrow()` interface, this will provide an inconsistent interface - different to the readr API we're trying to emulate.

We could rewrite the interface for `write_csv_arrow()` to map more closely to `readr::write_csv()` to enable these other changes. However, this would constitute a breaking change. It's unclear how much user impact this would have.

Changes needed to be made to `write_csv_arrow()`:
* expose additional options available through `CsvWriteOptions`:
* `quoting_style`
* `eol`
* `delimiter`
* add parameter `quote` to `write_csv_arrow()`
* collapse parameters `sink` and `file` into a single parameter (by checking the type of the object passed in and changing behavior accordingly)
* collapse parameters `include_header` and `col_names` into a single parameter (`col_names`)
* removing the `batch_size` param (which should be specified in `write_options`
* updating the default value of the `na` param to match `readr::write_csv`
* adding parameter `eol` which is mapped to that option
* potentially open an issue requesting the option to specify `escape` ("The type of escape to use when quotes are in the data") OR just not implementing this parameter OR implement it and set the default to whatever Arrow does by default and raise an error if the user specifies an alternative value

### Component(s)

R

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.