easystats / easystats/parameters
Inconsistency in p_adjust when working with emmeans/emGrid objects
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 499
- Forks
- 45
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 3
Description
Let's assume I have an emmeans/emGrid object already created:
> emg
Week emmean SE df asymp.LCL asymp.UCL
Week 1 -0.317 0.132 Inf -0.577 -0.0573
Week 2 -0.633 0.136 Inf -0.900 -0.3656
Week 4 -0.719 0.178 Inf -1.068 -0.3696
Week 8 -0.984 0.166 Inf -1.309 -0.6587
Week 16 -1.198 0.162 Inf -1.514 -0.8810
Week 56 -1.456 0.290 Inf -2.025 -0.8879
Covariance estimate used: vbeta
Confidence level used: 0.95
Now I want both CI + p-value to be adjusted simultaneously using the mutivariate t distribution:
> set.seed(1000); update(emg, infer = c(TRUE, TRUE), adjust="mvt")
Week emmean SE df asymp.LCL asymp.UCL z.ratio p.value
Week 1 -0.317 0.132 Inf -0.652 0.0181 -2.393 0.0717
Week 2 -0.633 0.136 Inf -0.977 -0.2880 -4.643 <.0001
Week 4 -0.719 0.178 Inf -1.170 -0.2681 -4.033 0.0003
Week 8 -0.984 0.166 Inf -1.404 -0.5642 -5.929 <.0001
Week 16 -1.198 0.162 Inf -1.606 -0.7889 -7.413 <.0001
Week 56 -1.456 0.290 Inf -2.190 -0.7227 -5.021 <.0001
Covariance estimate used: vbeta
Confidence level used: 0.95
Conf-level adjustment: mvt method for 6 estimates
P value adjustment: mvt method for 6 tests
Now I'm trying to do the same with parameters (expecting it will pass the parameter to emmeans since it understands this class):
> parameters(emg, p_adjust = "mvt")
Week | Marginal Means | SE | 95% CI | z | p
-----------------------------------------------------------------
Week 1 | -0.32 | 0.13 | [-0.58, -0.06] | -2.39 | 0.017
Week 2 | -0.63 | 0.14 | [-0.90, -0.37] | -4.64 | < .001
Week 4 | -0.72 | 0.18 | [-1.07, -0.37] | -4.03 | < .001
Week 8 | -0.98 | 0.17 | [-1.31, -0.66] | -5.93 | < .001
Week 16 | -1.20 | 0.16 | [-1.51, -0.88] | -7.41 | < .001
Week 56 | -1.46 | 0.29 | [-2.02, -0.89] | -5.02 | < .001
p-value adjustment method: mvt
Warning message:
`p_adjust` must be one of holm, hochberg, hommel, bonferroni, bh, by, fdr, none, tukey, scheffe, sidak
OK, let's try "bh"
> parameters(emg, p_adjust = "bh")
Error in match.arg(method) :
'arg' should be one of “holm”, “hochberg”, “hommel”, “bonferroni”, “BH”, “BY”, “fdr”, “none”
Also doesn't work. So maybe the "BH":
> parameters(emg, p_adjust = "BH")
Week | Marginal Means | SE | 95% CI | z | p
-----------------------------------------------------------------
Week 1 | -0.32 | 0.13 | [-0.58, -0.06] | -2.39 | 0.017
Week 2 | -0.63 | 0.14 | [-0.90, -0.37] | -4.64 | < .001
Week 4 | -0.72 | 0.18 | [-1.07, -0.37] | -4.03 | < .001
Week 8 | -0.98 | 0.17 | [-1.31, -0.66] | -5.93 | < .001
Week 16 | -1.20 | 0.16 | [-1.51, -0.88] | -7.41 | < .001
Week 56 | -1.46 | 0.29 | [-2.02, -0.89] | -5.02 | < .001
p-value adjustment method: Benjamini & Hochberg (1995)
This one worked.
So, to summarize:
- the "mvt" isn't passed to emmeans, even if parameters understand this class (but maybe it was not the goal - then I'm sorry for reporting it)
- but even if we pick the other options, the hint is given using all small letters
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the parameters() entry point for emmeans/emGrid objects and trace how p_adjust is validated and passed through. Compare the handling of "mvt", lowercase "bh", and uppercase "BH" with the behavior shown in the examples. Done means supported adjustment names are handled consistently and the requested emmeans adjustment behavior is preserved or clearly reported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- analytics
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100