Using kfold for model selection after splitting by groups
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 157
- Forks
- 38
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 2
Description
Using packages rstanarm and loo to run a logistic regression with four different intercepts and univariate slope hierarchical by individual, then using k-fold leave-one-group-out for model selection (kfold_split_grouped, then kfold). I get the same error when running the kfold command regardless of the model complexity.
Error message:
Fitting K = 60 models distributed over 3 cores
Error in checkForRemoteErrors(val) :
3 nodes produced errors; first error: object 'n_chains' not found
I think it has to do with the number of cores in the kfold command (The above error occurred with 3 cores). If I run it with one core, it works but impossibly slowly. More than one core, and there is a problem.
I updated R and all packages yesterday:
R Session info:
R version 4.0.5 (2021-03-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
other attached packages:
loo_2.4.1 MCMCvis_0.15.1 rstanarm_2.21.1 Rcpp_1.0.6
I have included some ineligant sample code that should reproduce the error. The problem occurs at line 83.
rstanarm_logistic_hier_test.txt
Thanks,
LKS
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the attached rstanarm_logistic_hier_test.txt sample and line 83, then reproduce the kfold_split_grouped and kfold calls using three cores and one core. Compare the parallel error involving n_chains with the successful single-core run; done means grouped k-fold model selection runs correctly with multiple cores.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100