Add hierarchical stacking
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 157
- Forks
- 38
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 2
Description
Given the new research progress on "hierarchical stacking", it would be useful to implement this new tool into the loo package.
This new tool needs more user customization: after a few models are fit, the user needs to
- Run pointwise model evaluation. For IID data, this is highly automated.
- Specify what variable that stacking weight should depend on. This cannot be automated.
- Come up with a stacking model. A template may look like https://github.com/yao-yl/hierarchical-stacking-code/blob/main/well_switch/optim.stan. But this model could be more complicated too.
- Specify hyper prior.
The challenge for putting this package into the present package lies in that
- Extra dependence on stan.
- May even need stanglm/brms syntax to write stacking models.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the loo package and the linked hierarchical-stacking-code template, especially well_switch/optim.stan. Determine how pointwise model evaluation, the user-selected stacking variable, the stacking model, and its hyperprior would fit with the proposed Stan and stanglm/brms dependencies. Done would mean a defined implementation path for this customizable hierarchical stacking tool.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 18/100