Moment matching with iwmm package
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 157
- Forks
- 38
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 2
Description
Currently loo_moment_match only works on stanfit objects (due to the reliance on the log_prob methods from rstan), but not CmdStanFit or just a matrix of draws (see e.g. #209). I was thinking about the best way to expand support of loo_moment_match to other objects, and am opening this issue mostly for discussion at this stage.
@topipa has created a generic implementation of moment matching (iwmm) which works on a matrix or stanfit object. Recently we have also added CmdStanFit support to this package (using the new model methods in cmdstanr which are currently not yet in a release).
iwmm is not (yet) on CRAN, so now might be a good time to discuss the best way to interoperate with loo.
After some discussion with @avehtari, I currently see three options (there may be others I haven't thought of):
- update functions in loo, no change to dependencies
Code from iwmm is copied into loo and adapted for use inloo::loo_moment_match(made specific for leave-one-out importance posteriors). iwmm would remain an independent package for generic importance sampling. - add iwmm dependency in loo
iwmm is submitted to CRAN andloo::loo_moment_matchis rewritten to useiwmm::moment_match. loo would then import and depend on iwmm. - move iwmm functions into loo
All iwmm functions are moved into loo, and kept generic (i.e. not specific for leave-one-out posteriors). Keeping the functions generic has some precedence as loo is the home ofloo::psiswhich is used in cases other than leave-one-out CV (e.g. in priorsense and adjustr).
@jgabry @avehtari @topipa @paul-buerkner , do you have any thoughts on this?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing loo::loo_moment_match and the generic implementation in the linked iwmm package, including its matrix, stanfit, and CmdStanFit support. Compare the three interoperability options in the issue and determine which design should be adopted; done means the project has a decided approach for expanding supported inputs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100