stan-dev / stan-dev/loo

Add support functions for Mixture IS LOO

Open
#270 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
157
Forks
38
Avg merge
4d 16h
Merged PRs (30d)
2

Description

We have the Mix-IS-LOO vignette https://mc-stan.org/loo/articles/loo2-mixis.html, but it would be useful to add support for creating a loo object from mixture posterior log_lik. Assuming the Stan model code implements the mixture distribution and generates log_lik variable, Mix-IS-LOO elpd_loo estimate is given by

log_lik_mix <- extract(fit_mix)$log_lik
l_common_mix <- rowLogSumExps(-log_lik_mix)
log_weights <- -log_lik_mix - l_common_mix
elpd_mixis <- logSumExp(-l_common_mix) - rowLogSumExps(t(log_weights))

We can also compute Pareto-k's for each pointwise elpd, but there is a difference that the mixture posterior elpd itself is already different from the regular posterior, so the error of Mix-IS-LOO tend to be smaller than regular PSIS-LOO.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the Mix-IS-LOO vignette at https://mc-stan.org/loo/articles/loo2-mixis.html and use the issue's log_lik_mix, l_common_mix, log_weights, and elpd_mixis calculations as the behavioral reference. Done should provide support for creating a loo object from mixture posterior log_lik and compute Pareto-k values where applicable.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
analytics, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.