Safe priors misclassify no-intercept categorical cell means as slopes
- Dominant language
- Python
- Stars
- 124
- Forks
- 24
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 60
Description
Hello, I'm working with the HSSM package (v 0.2.5) and encountering an issue when using my own data. Specifically, when using models where the boundary parameter `a` changes across conditions *without* an intercept (see code snippet 1 below), the sampling process terminates prematurely, and the resulting samples are mostly invalid, leading to NaN values for r_hat. However, when I use models where the boundary parameter `a` changes across conditions *with* an intercept (see code snippet 2 below), the sampling process appears more normal, and the sampling results are acceptable. Could you please investigate this issue?
```
#snippet 1
study1_model_reg_vaz = hssm.HSSM(
data = study1_RTdata,
model = 'ddm',
prior_settings = "safe",
#link_settings = "log_logit",
#loglik_kind = "approx_differentiable",
include = [
{
"name": "v",
"formula": "v ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
{
"name": "z",
"formula": "z ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
{
"name": "a",
"formula": "a ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
],
)
%%time
idata_study1_model_reg_vaz = study1_model_reg_vaz.sample(
#sampler = "mcmc",
sampler = "nuts_numpyro", #nuts_numpyro
chains = 2,
cores = 4,
draws = 1000,
tune = 500,
#target_accept = 0.95,
#idata_kwargs = dict(log_likelihood = True), # return log likelihood
)
```
RuntimeWarning: invalid value encountered in scalar divide (between_chain_variance [/](https://file+.vscode-resource.vscode-cdn.net/) within_chain_variance + num_samples - 1) [/](https://file+.vscode-resource.vscode-cdn.net/) (num_samples) There were 2000 divergences after tuning. Increase `target_accept` or reparameterize. We recommend running at least 4 chains for robust computation of convergence diagnostics 100%|██████████| 2000/2000 [00:20<00:00, 99.03it/s]
CPU times: user 2min 15s, sys: 13.2 s, total: 2min 28s Wall time: 44.6 s

```
#snippet 2
study1_model_reg_vaz = hssm.HSSM(
data = study1_RTdata,
model = 'ddm',
prior_settings = "safe",
#link_settings = "log_logit",
#loglik_kind = "approx_differentiable",
include = [
{
"name": "v",
"formula": "v ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
{
"name": "z",
"formula": "z ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
{
"name": "a",
"formula": "a ~ 1 + stimulus_set:target_race:target_expression",
"link": "identity",
},
],
)
%%time
idata_study1_model_reg_vaz = study1_model_reg_vaz.sample(
#sampler = "mcmc",
sampler = "nuts_numpyro", #nuts_numpyro
chains = 2,
cores = 4,
draws = 1000,
tune = 500,
#target_accept = 0.95,
#idata_kwargs = dict(log_likelihood = True), # return log likelihood
)
```
Contributor guide
Research direction
Start by reproducing the two HSSM.HSSM configurations using prior_settings="safe" and the differing a formulas, then inspect the safe-prior handling for no-intercept categorical terms. Compare sampling diagnostics between the configurations; done means no premature termination, invalid samples, or NaN r_hat values for the no-intercept model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100