stan-dev / stan-dev/loo

Add hierarchical stacking

Open
#174 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
R
Stars
157
Forks
38
Avg merge
4d 16h
Merged PRs (30d)
2

Description

Given the new research progress on "hierarchical stacking", it would be useful to implement this new tool into the loo package.

This new tool needs more user customization: after a few models are fit, the user needs to

  1. Run pointwise model evaluation. For IID data, this is highly automated.
  2. Specify what variable that stacking weight should depend on. This cannot be automated.
  3. Come up with a stacking model. A template may look like https://github.com/yao-yl/hierarchical-stacking-code/blob/main/well_switch/optim.stan. But this model could be more complicated too.
  4. Specify hyper prior.

The challenge for putting this package into the present package lies in that

  1. Extra dependence on stan.
  2. May even need stanglm/brms syntax to write stacking models.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the loo package and the linked hierarchical-stacking-code template, especially well_switch/optim.stan. Determine how pointwise model evaluation, the user-selected stacking variable, the stacking model, and its hyperprior would fit with the proposed Stan and stanglm/brms dependencies. Done would mean a defined implementation path for this customizable hierarchical stacking tool.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.