Efficient saving/reading of data frames containing rvars?
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 171
- Forks
- 26
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 3
Description
Thank you so much for this fantastic package, especially the rvars data type which I really enjoy using. I've recently run into a situation where I would like to save a data frame (about 700 rows by 10 variables) that contains a single 1d rvar column from one script and read it in for use in another script. When I try to save the data frame with saveRDS, however, it takes 6+ minutes to complete the operation and the resulting object, which is ~20 MB in R, is 1.5+ GB in size. While this is workable, I was wondering if there are know solutions or recommended alternatives to saveRDS when working with rvars in this way. Thank you!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported saveRDS behavior with a roughly 700-row data frame containing a single 1d rvar column, measuring time and serialized size. Start by tracing how rvars are represented during saveRDS and compare possible save/read approaches. Done means documenting or implementing a substantially more efficient way to save and reload this data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100