stan-dev / stan-dev/rstan

thin or get less iteration of only certain parameters (request)

Open
#398 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
1.1k
Forks
266
Avg merge
2h 56m
Merged PRs (30d)
1

Description

Summary:

It would be nice to be able to get less samples of some parameters (e.g. predictions or less log-likelihoods from generated quantities section) to avoid models of more than 4GB.

Description:

I usually run a model with 2000 iterations and 4 chains. I have predicted values, and the log likelihood of every observation in my generated quantities, but I don't really need 4000 samples of each one.
And if I don't exclude them when I fit a model with stan(...) and I end up with a model of more than 4gb. (As you know R is not very friendly to huge files).

(Edited:Ok, I see now, that it's possible to save directly a huge csv file, without loading all the samples in memory. It still would be nice to be able to split the file between the main parameters, and those that clog R like likelihoods or predicted values, or to only save them)

Reproducible Steps:

Just using model <- stan(...) without excluding parameters

Current Output:

A model with all the samples of all the parameters > 1gb.

Expected Output:

A model with less the samples of some of the parameters.

RStan Version:

2.14.1

R Version:

3.3.2 (2016-10-31)

Operating System:

Ubuntu 16.04

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from the stan(...) call described in the issue and trace how generated-quantity predictions and log-likelihood samples are retained in the fitted model. Clarify the intended selection and sampling behavior, then define tests showing that selected parameters can be stored with fewer samples while the remaining model output is preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.