Feature: heteroscedastic Tweedie loss
- Dominant language
- C++
- Stars
- 9.1k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
CatBoost has currently a limited number of built-in heteroscedastic loss functions. `RMSEWithUncertainty` is implemented for heteroscedastic Gaussian noise, but some domains, such as insurance, often deal with Tweedie-distributed data where the dispersion parameter ϕ isn't constant across observations. A heteroscedastic Tweedie loss function could be useful in any machine learning problem that models zero-inflated nonnegative continuous data.
## Proposed Approach:
- Predict two values: z1=log(μ) and z2=log(ϕ). The power parameter p will be passed as a constant hyperparameter as is done in the current homoscedastic Tweedie implementation.
- Use the saddlepoint approximation for the log-likelihood to derive the gradients and Hessians for both parameters, as detailed in [Dunn & Smyth (2005)](https://gksmyth.github.io/pubs/tweediepdf-series-preprint.pdf).
- Implement this in the C++ backend.
I am happy to implement this myself. I propose `TweedieWithUncertainty` as the loss name, analogously to the RMSE equivalent.
Contributor guide
Research direction
Start by locating CatBoost's existing homoscedastic Tweedie implementation and the RMSEWithUncertainty loss in the C++ backend. Read Dunn & Smyth (2005) to understand the proposed saddlepoint approximation and derive the gradients and Hessians for z1 and z2. Done means a loss named TweedieWithUncertainty that accepts p as a constant hyperparameter and supports heteroscedastic Tweedie modeling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100