easystats / easystats/performance
`r2_nakagawa()` is **NOT** location invariant when COV is not modelled (and is wrong?)
Open
Nobody has claimed this yet.
3 investigators :grey_question::question:
Bug :bug:
- Dominant language
- R
- Stars
- 1.2k
- Forks
- 109
- Avg merge
- 6h 34m
- Merged PRs (30d)
- 8
Description
This is some weird stuff.
library(lme4)
#> Loading required package: Matrix
library(performance)
r2_nakagawa2 <- function(model) {
y <- insight::get_response(model)
c(
Cond. = cor(predict(model, re.form = NULL), y)^2,
Marg. = cor(predict(model, re.form = NA), y)^2
)
}
# With COV =================================
m1 <- lmer(Reaction ~ Days + (Days|Subject),
data = sleepstudy)
# Change location -------
sleepstudy$Days0 <- sleepstudy$Days - 10
m2 <- lmer(Reaction ~ Days0 + (Days0|Subject),
data = sleepstudy)
#> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
#> Model failed to converge with max|grad| = 0.00422952 (tol = 0.002, component 1)
r2_nakagawa(m1)
#> # R2 for Mixed Models
#>
#> Conditional R2: 0.799
#> Marginal R2: 0.279
r2_nakagawa(m2) # same
#> # R2 for Mixed Models
#>
#> Conditional R2: 0.799
#> Marginal R2: 0.279
# Validate:
r2_nakagawa2(m1)
#> Cond. Marg.
#> 0.8271703 0.2864714
r2_nakagawa2(m2)
#> Cond. Marg.
#> 0.8271702 0.2864714
# Without COV =================================
m1b <- lmer(Reaction ~ Days + (Days||Subject),
data = sleepstudy)
# Change location -------
m2b <- lmer(Reaction ~ Days0 + (Days0||Subject),
data = sleepstudy)
r2_nakagawa(m1b) # different
#> # R2 for Mixed Models
#>
#> Conditional R2: 0.702
#> Marginal R2: 0.415
r2_nakagawa(m2b) # even more different
#> # R2 for Mixed Models
#>
#> Conditional R2: 0.864
#> Marginal R2: 0.183
r2_nakagawa2(m1b) # same as with cov
#> Cond. Marg.
#> 0.8279419 0.2864714
r2_nakagawa2(m2b) # same as with cov
#> Cond. Marg.
#> 0.8244819 0.2864714
Created on 2022-06-03 by the reprex package (v2.0.1)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the r2_nakagawa() entry point and reproduce the lme4 sleepstudy examples in the issue, comparing models with and without covariance and shifted Days values. Determine the intended location-invariant result for models where COV is not modelled; done means the behavior is corrected and covered by an appropriate regression check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100