lnccbrown / lnccbrown/HSSM

Safe priors misclassify no-intercept categorical cell means as slopes

Open
#720 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
124
Forks
24
Avg merge
19h 32m
Merged PRs (30d)
60

Description

Hello, I'm working with the HSSM package (v 0.2.5) and encountering an issue when using my own data. Specifically, when using models where the boundary parameter `a` changes across conditions *without* an intercept (see code snippet 1 below), the sampling process terminates prematurely, and the resulting samples are mostly invalid, leading to NaN values for r_hat. However, when I use models where the boundary parameter `a` changes across conditions *with* an intercept (see code snippet 2 below), the sampling process appears more normal, and the sampling results are acceptable. Could you please investigate this issue?

```
#snippet 1
study1_model_reg_vaz = hssm.HSSM(
data = study1_RTdata,
model = 'ddm',
prior_settings = "safe",
#link_settings = "log_logit",
#loglik_kind = "approx_differentiable",
include = [
{
"name": "v",
"formula": "v ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
{
"name": "z",
"formula": "z ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
{
"name": "a",
"formula": "a ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},

],
)

%%time
idata_study1_model_reg_vaz = study1_model_reg_vaz.sample(
#sampler = "mcmc",
sampler = "nuts_numpyro", #nuts_numpyro
chains = 2,
cores = 4,
draws = 1000,
tune = 500,
#target_accept = 0.95,
#idata_kwargs = dict(log_likelihood = True), # return log likelihood
)

```

RuntimeWarning: invalid value encountered in scalar divide (between_chain_variance [/](https://file+.vscode-resource.vscode-cdn.net/) within_chain_variance + num_samples - 1) [/](https://file+.vscode-resource.vscode-cdn.net/) (num_samples) There were 2000 divergences after tuning. Increase `target_accept` or reparameterize. We recommend running at least 4 chains for robust computation of convergence diagnostics 100%|██████████| 2000/2000 [00:20<00:00, 99.03it/s]

CPU times: user 2min 15s, sys: 13.2 s, total: 2min 28s Wall time: 44.6 s

![](https://cdn.jsdelivr.net/gh/yuanbobq/images@master/img/202504242145370.png)

```
#snippet 2
study1_model_reg_vaz = hssm.HSSM(
data = study1_RTdata,
model = 'ddm',
prior_settings = "safe",
#link_settings = "log_logit",
#loglik_kind = "approx_differentiable",
include = [
{
"name": "v",
"formula": "v ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
{
"name": "z",
"formula": "z ~ 0 + stimulus_set:target_race:target_expression",
"link": "identity",
},
{
"name": "a",
"formula": "a ~ 1 + stimulus_set:target_race:target_expression",
"link": "identity",
},

],
)

%%time
idata_study1_model_reg_vaz = study1_model_reg_vaz.sample(
#sampler = "mcmc",
sampler = "nuts_numpyro", #nuts_numpyro
chains = 2,
cores = 4,
draws = 1000,
tune = 500,
#target_accept = 0.95,
#idata_kwargs = dict(log_likelihood = True), # return log likelihood
)

```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the two HSSM.HSSM configurations using prior_settings="safe" and the differing a formulas, then inspect the safe-prior handling for no-intercept categorical terms. Compare sampling diagnostics between the configurations; done means no premature termination, invalid samples, or NaN r_hat values for the no-intercept model.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.