stan_rdump fails for large arrays

Summary:

I'm using stan_rdump to write out a stan data Rds file in order to fit a model to moderately large data related to genomics. I can't fit this model using rstan because long vectors aren't supported yet by mclapply, however I would like to write out the data using stan_rdump to fit the model using cmdstan.

However it looks like the data are too large to write out to an Rds file:

rstan::stan_rdump(large_standata)
> Error in paste(as.vector(vv), collapse = ", ") : 
  result would exceed 2^31-1 bytes

For context, my large_standata object looks like the following when generated by brms:

> str(large_standata)
List of 23
 $ N              : int 139368
 $ Y              : num [1:139368(1d)] 4 0 0 0 25 0 0 18 0 0 ...
 $ K              : int 23229
 $ X              : num [1:139368, 1:23229] 1 1 1 1 1 1 1 1 1 1 ...
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : chr [1:139368] "1" "2" "3" "4" ...
  .. ..$ : chr [1:23229] "Intercept" "treatment_status" "treatment_status:gene41698" "treatment_status:gene41699" ...
  ..- attr(*, "assign")= int [1:23229] 0 1 2 2 2 2 2 2 2 2 ...
  ..- attr(*, "contrasts")=List of 1
  .. ..$ gene: chr "contr.treatment"
 $ Z_1_1          : num [1:139368(1d)] 1 1 1 1 1 1 1 1 1 1 ...
  ..- attr(*, "dimnames")=List of 1
  .. ..$ : chr [1:139368] "1" "2" "3" "4" ...
 $ Z_2_1          : num [1:139368(1d)] 1 1 1 1 1 1 1 1 1 1 ...
  ..- attr(*, "dimnames")=List of 1
  .. ..$ : chr [1:139368] "1" "2" "3" "4" ...
 $ hs_df          : num 3
 $ hs_df_global   : num 1
 $ hs_df_slab     : num 4
 $ hs_scale_global: num 0.000268
 $ hs_scale_slab  : num 2
 $ K_zi           : int 1
 $ X_zi           : num [1:139368, 1] 1 1 1 1 1 1 1 1 1 1 ...
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : chr [1:139368] "1" "2" "3" "4" ...
  .. ..$ : chr "Intercept"
  ..- attr(*, "assign")= int 0
 $ Z_1_zi_2       : num [1:139368(1d)] 1 1 1 1 1 1 1 1 1 1 ...
  ..- attr(*, "dimnames")=List of 1
  .. ..$ : chr [1:139368] "1" "2" "3" "4" ...
 $ J_1            : int [1:139368(1d)] 22141 6013 6008 14929 11181 10096 10088 10121 14923 14926 ...
 $ N_1            : int 23228
 $ M_1            : int 2
 $ NC_1           : num 1
 $ J_2            : int [1:139368(1d)] 1 1 1 1 1 1 1 1 1 1 ...
 $ N_2            : int 6
 $ M_2            : int 1
 $ NC_2           : num 0
 $ prior_only     : int 0
 - attr(*, "class")= chr "standata"

Then problem seems to arise because stan_rdump constructs a character representation of the rdump file & that exceeds the R memory limits.

Description:

It would be nice to be able to write these data out to file in order to fit a (large-memory &/or long-running) model using cmdstan.

Reproducible Steps:

See above for a description of steps. I have posted a reproducible example in a github repo.

Current Output:

result would exceed 2^31-1 bytes

Expected Output:

Writing the Rds to disk

RStan Version:

2.18.2

R Version:

"R version 3.5.1 (2018-07-02)"

Operating System:

Ubuntu/Debian stretch (x86_64-pc-linux-gnu (64-bit))

Start with the stan_rdump entry point and the linked reproducible example; inspect how the large object is converted before writing. Done means large standata can be written for CmdStan without the result-would-exceed-2^31-1-bytes failure.