user defined loss/utility functions
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 157
- Forks
- 38
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 2
Description
Section 4.3 of https://avehtari.github.io/modelselection/diabetes.html demonstrates how to use loo package with other loss/utility function than log score. LOO-R^2 is another example. Both of these are just for point predictions, which is the easiest case.
Make it easier to provide user defined loss/utility function for point predictions. Arguments would be
- point estimate type given for E_loo (e.g. "mean")
- user defined function with at least two arguments 1) point predictions and 2) data y
We can later consider expanding to loo/utility functions for predictive distributions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Section 4.3 of the linked diabetes example and inspect the loo package's existing point-prediction and E_loo entry points. Define how callers provide the point-estimate type and a loss or utility function receiving predictions and y; done means point-prediction functions can use this interface without addressing predictive distributions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100