apache / apache/arrow-cookbook
[R] Recipe for random sampling
- Dominant language
- C++
- Stars
- 108
- Forks
- 49
- Avg merge
- 2h 14m
- Merged PRs (30d)
- 1
Description
It would be great if there were a way to sample from an arrow dataset. I put together this somewhat hacky example, but I bet there's some thing a bit more elegant..
```
library(arrow)
library(dplyr)
library(nycflights13)
flights <- nycflights13::flights
flights$id <- seq_len(nrow(flights))
for(i in unique(flights$month)) {
out <- filter(flights, month == i)
arrow::write_parquet(out, paste0("flight_ds/", i, ".parquet"))
}
ds <- arrow::open_dataset("flight_ds")
sample <- sample(flights$id, 100)
ds %>%
filter(id %in% sample) %>%
collect()
```
Contributor guide
Research direction
Start by reviewing the R example in the issue and the existing cookbook structure to determine where an arrow dataset sampling recipe belongs. Establish an elegant, documented workflow for sampling from the dataset, and confirm that the recipe is reproducible and clearly improves on the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100