apache / apache/arrow

[R] write_* methods can't have socketConnection as a sink

Open
#34,779 3 comments 0 reactions 0 assignees View on GitHub
Component: R Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Describe the enhancement requested

I am trying to make a straight write to a raw socket replicating the python's implementation.
For some reason yet unknown to me, sending a raw vector combining `arrow::write_to_raw(table)` and `writeBin()` but doesn't seem to be acceptable. And trying to send files by batches is not possible since it doesn't allow me to write to sockets directly, only to `OutputStream`, which doesn't allow socketConnection as well.

```r
rbIris <- record_batch(iris)

nDriver <- socketConnection(host = '127.0.0.1',
port = 1234,
blocking = TRUE,
server = FALSE,
open = "w+b")

writer <- arrow::RecordBatchStreamWriter$create(nDriver , rbIris$schema)
```

`Error: sink must be a "OutputStream"`

### Component(s)

R

Contributor guide

Open the contributing guide

Research direction

Start at the R RecordBatchStreamWriter$create call and trace how its sink is validated against OutputStream. Compare the socketConnection and write_to_raw/writeBin examples with the existing Python behavior described in the issue. Done means R batch writes can use a socketConnection directly, with the demonstrated socket example working without the OutputStream error.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data-engineering, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.