stan-dev / stan-dev/rstan

Error with "unserialize(node$con) : error reading from connection"

Open
#243 34 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
1.1k
Forks
266
Avg merge
2h 56m
Merged PRs (30d)
1

Description

EDIT: The manifestation of this problem has morphed over time. Currently, models won't run with large data sets at all, even if on 1 core.

I only get this error when I'm trying to analyze the full version of my data set using multiple cores. If I analyze the full data set on 1 core (1 chain, 4 iterations), it runs. Similarly, if I run the model for a very small fraction of the data set on multiple cores (4 cores, 4 chains, 4 iterations), it also runs fine.

Below are the errors for a remote Linux machine that had ~133 GB RAM available at the time. The schools example runs fine over there (both on 1 core, and 4, and 10). However, I also get similar errors on my local Mac. Sometimes I get the same error message, other times R just crashes (the console just goes away; this is not a 'hang'!). Looking at the resources, neither system appears to be having any trouble lead up to the crash/ error.

starting worker pid=34649 on localhost:11567 at 17:34:31.565
starting worker pid=34658 on localhost:11567 at 17:34:31.798
starting worker pid=34667 on localhost:11567 at 17:34:32.032
starting worker pid=34676 on localhost:11567 at 17:34:32.268

SAMPLING FOR MODEL 'msomStatic' NOW (CHAIN 1).

SAMPLING FOR MODEL 'msomStatic' NOW (CHAIN 2).

SAMPLING FOR MODEL 'msomStatic' NOW (CHAIN 3).

SAMPLING FOR MODEL 'msomStatic' NOW (CHAIN 4).
Error in unserialize(node$con) : error reading from connection
Calls: stan ... FUN -> recvData -> recvData.SOCK0node -> unserialize

Session info for the Linux:

> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS release 6.6 (Final)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] rstan_2.8.2            ggplot2_1.0.1          rbLib_0.0.2
[4] trawlDiversity_0.0.0.9 trawlData_0.1.0        data.table_1.9.6
[7] devtools_1.9.1

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.2      magrittr_1.5     MASS_7.3-43      munsell_0.4.2
 [5] colorspace_1.2-6 stringr_1.0.0    plyr_1.8.3       tools_3.2.2
 [9] grid_3.2.2       gtable_0.1.2     digest_0.6.8     gridExtra_2.0.0
[13] reshape2_1.4.1   numbers_0.6-1    memoise_0.2.1    inline_0.3.14
[17] stringi_1.0-1    ssh.utils_1.0    scales_0.3.0     stats4_3.2.2
[21] chron_2.3-47     proto_0.3-10

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported rstan sampling failure with the full data set on one and multiple cores, using the supplied R, CentOS, and rstan 2.8.2 session details. Compare the worker-process behavior behind the unserialize(node$con) error and R crashes; done means the failure is isolated and a verified fix or clear limitation is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.