apache / apache/arrow-cookbook

[R] Recipe for random sampling

Open
#83 14 comments 0 reactions 0 assignees View on GitHub
r
Dominant language
C++
Stars
108
Forks
49
Avg merge
2h 14m
Merged PRs (30d)
1

Description

It would be great if there were a way to sample from an arrow dataset. I put together this somewhat hacky example, but I bet there's some thing a bit more elegant..

```
library(arrow)
library(dplyr)
library(nycflights13)

flights <- nycflights13::flights

flights$id <- seq_len(nrow(flights))

for(i in unique(flights$month)) {
out <- filter(flights, month == i)
arrow::write_parquet(out, paste0("flight_ds/", i, ".parquet"))
}

ds <- arrow::open_dataset("flight_ds")

sample <- sample(flights$id, 100)

ds %>%
filter(id %in% sample) %>%
collect()
```

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the R example in the issue and the existing cookbook structure to determine where an arrow dataset sampling recipe belongs. Establish an elegant, documented workflow for sampling from the dataset, and confirm that the recipe is reproducible and clearly improves on the example.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.