Failure to dump large array with "stan_rdump"
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 1.1k
- Forks
- 266
- Avg merge
- 2h 56m
- Merged PRs (30d)
- 1
Description
Summary:
Failure to allocate memory when dumping large array with function stan_rdump.
Description:
I tried to dump results extracted from a stanfit object into file using stan_rdump function. When I do so, I encountered an error:
Error in gsub(addnlpat, "\1\n", str) :
'Calloc' could not allocate memory (18446744071562067968 of 1 bytes)
I thought the problem is that the array is too large to dump (77112000 elements, 588.3Mb). Hope someone could rewrite the stan_rdump function to work with large objects.
e.g. something like this:
rdump <- function (list, file = "", envir=parent.frame())
{
file <- file(file, "a")
for (v in list) {
vv <- get(v, envir)
vvdim <- dim(vv)
cat(v, " <- \n", file = file, sep = "")
if (length(vv) == 0) {
str <- paste0("structure(integer(0), ")
}
else {
str <- paste0("structure(c(", paste(as.vector(vv),
collapse = ", "), "),")
}
cat(str, "\n", ".Dim = c(", paste(vvdim, collapse = ", "), "))\n",
file = file,
sep = "")
}
}
RStan Version:
2.19.2
R Version:
The version of R you are running (e.g., from R.version.string)
R version 3.6.1 (2019-07-05)
Operating System:
macOS 10.15.3
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the stan_rdump entry point and reproduce the failure with the reported 77,112,000-element array on the stated R and RStan versions. Trace the allocation involved in constructing and writing the dump; done means a large array can be dumped without the reported Calloc error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100