problems using loo in clogit models
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 157
- Forks
- 38
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 2
Description
In a rstanarm branch, I introduced a stan_clogit function which is similar to the clogit function in the survival package but is actually described in more detail by Stata ( http://www.stata.com/manuals13/rclogit.pdf ). So, you will be able to do
post <- stan_clogit(case ~ spontaneous + induced, strata = stratum,
data = infert[order(infert$stratum, !infert$case),], QR = TRUE)
The problem is that some groups have the same values on the dummy variables spontaneous and induced, such as
> infert[infert$stratum == 3,]
education age parity induced case spontaneous stratum pooled.stratum
3 0-5yrs 39 6 2 1 0 3 4
86 0-5yrs 39 6 2 0 0 3 4
168 0-5yrs 39 6 2 0 0 3 4
Thus, no matter what are the posterior realizations of the coefficients on spontaneous and induced, the likelihood for group 3 is the same.
I am pretty sure the correct concept for a clogit model is to imagine leaving out one group rather than one observation, but when I call loo::loo.matrix, the third column of the input is a constant, which causes the Pareto k estimate to be infinite. Since the third group could be omitted and only change the log-likelihood by a constant, this seems to be unreasonable.
The question becomes, what function should be catching this? We could have stan_clogit drop groups that have only one unique row in the design matri{x,ces}. We could have loo.stanreg omit such groups. Or loo::loo could check which Pareto k estimates are infinite and change them to some number when the log likelihoods are finite but constant. Thoughts @avehtari ?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the constant third-column Pareto-k behavior with loo::loo.matrix using the stan_clogit example and infert data described in the issue. Review how loo::loo, loo::loo.matrix, and loo.stanreg divide responsibility, then resolve which behavior is intended and define regression coverage for the agreed handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100