lnccbrown / lnccbrown/ssm-simulators
Custom `boundary` is accepted and silently ignored by 13 models whose compiled engine has no `boundary_fun` argument
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 24
- Forks
- 18
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 9
Description
`Simulator(model, boundary=fn, boundary_params=[...])` builds a boundary dict and splats it into the compiled engine unconditionally. Eight `cssm` engines have no `boundary_fun` parameter, so the boundary is swallowed by `**kwargs` and discarded; the run completes normally using the engine's built-in boundary. 11 of the 13 affected models emit no warning at all, and for `ddm_legacy`/`addm` the returned `metadata["boundary_fun_type"]` reports the built-in boundary name. 97 of 113 built-in models honour a custom boundary correctly, and default usage (no boundary override) is unaffected.
Environment: ssm-simulators 0.13.2, Python 3.12, macOS (Darwin 25.4.0), `n_threads=1`.
## Reproducer
```python
"""ssms 0.13.2 -- custom boundary silently ignored by several compiled engines."""
import inspect
import numpy as np
from ssms.basic_simulators.simulator_class import Simulator
from ssms.config import get_model_registry
# Collapses to the 0.01 floor within ~100 ms. Any engine that honours it
# must return near-instant RTs.
def hard_collapse(t, a=1.0):
return np.maximum(a * np.exp(-50.0 * np.asarray(t)), 0.01)
def mean_rt(model, theta, **kw):
out = Simulator(model, **kw).simulate(theta=theta, n_samples=2000, max_t=10.0)
rts = np.asarray(out["rts"]).ravel()
return float(rts[rts != -999.0].mean())
BND = dict(boundary=hard_collapse, boundary_params=["a"])
ddm_th = {"v": 0.5, "a": 1.5, "z": 0.5, "t": 0.0}
lba_th = {"v0": 0.5, "v1": 0.3, "v2": 0.2, "a": 1.5, "z": 0.2, "theta": 0.0}
print("ddm default %.3f -> hard_collapse %.3f (honoured)"
% (mean_rt("ddm", ddm_th), mean_rt("ddm", ddm_th, **BND)))
print("ddm_legacy default %.3f -> hard_collapse %.3f (IGNORED)"
% (mean_rt("ddm_legacy", ddm_th), mean_rt("ddm_legacy", ddm_th, **BND)))
print("lba_angle_3 default %.3f -> hard_collapse %.3f (IGNORED)"
% (mean_rt("lba_angle_3", lba_th), mean_rt("lba_angle_3", lba_th, **BND)))
print("lba_angle_3 control: same model, a=1.5 -> a=0.3 gives %.3f "
"(engine does react to 'a', just not to boundary_fun)"
% mean_rt("lba_angle_3", {**lba_th, "a": 0.3}))
# ---- sweep every built-in model -------------------------------------------
ignored = []
for m in sorted(get_model_registry().list_models()):
cfg = get_model_registry().get(m)
th = dict(zip(cfg["params"], map(float, cfg["default_params"])))
base, coll = mean_rt(m, th), mean_rt(m, th, **BND)
if not (base - coll) / base > 0.10:
ignored.append(m)
print("\nboundary silently ignored by %d/%d built-in models:" % (
len(ignored), len(get_model_registry().list_models())))
print(ignored)
print("\nengine signatures:")
for f in ["ddm_flexbound", "ddm", "addm", "lba_vanilla", "lba_angle",
"rlwm_lba_pw_v1", "rlwm_lba_race", "poisson_race",
"racing_diffusion_model"]:
import cssm
print(" cssm.%-24s boundary_fun in signature: %s"
% (f, "boundary_fun" in inspect.signature(getattr(cssm, f)).parameters))
cfg = get_model_registry().get("lba_angle_3")
print("\nlba_angle_3 config: boundary_name=%r boundary=%r boundary_params=%r"
% (cfg["boundary_name"], cfg["boundary"].__name__, cfg.get("boundary_params")))
```
## Output
```
ddm default 1.889 -> hard_collapse 0.046 (honoured)
ddm_legacy default 0.556 -> hard_collapse 0.562 (IGNORED)
lba_angle_3 default 2.863 -> hard_collapse 2.872 (IGNORED)
lba_angle_3 control: same model, a=1.5 -> a=0.3 gives 0.379 (engine does react to 'a', just not to boundary_fun)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['beta', 'q0', 'q1', 'deadline', 's']). The boundary function may receive missing arguments.
boundary_dict = make_boundary_dict(model_config_local, theta)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['beta', 'q0', 'q1', 'q2', 'deadline', 's']). The boundary function may receive missing arguments.
boundary_dict = make_boundary_dict(model_config_local, theta)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['beta', 'q0', 'q1', 'q2', 'q3', 'deadline', 's']). The boundary function may receive missing arguments.
boundary_dict = make_boundary_dict(model_config_local, theta)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['r1', 'r2', 'k1', 'k2', 't', 'deadline', 's', 'r', 'k']). The boundary function may receive missing arguments.
boundary_dict = make_boundary_dict(model_config_local, theta)
/…/site-packages/ssms/basic_simulators/simulator_class.py:651: UserWarning: Callable boundary expects parameters ['a'] but none were found in theta (keys: ['v0', 'v1', 'v2', 'A', 'b', 't', 'deadline', 's', 'v']). The boundary function may receive missing arguments.
boundary_dict = make_boundary_dict(model_config_local, theta)
boundary silently ignored by 16/113 built-in models:
['addm', 'ddm_legacy', 'dev_rlwm_lba_pw_v1', 'dev_rlwm_lba_race_v1', 'dev_rlwm_lba_race_v2', 'inv_temp_softmax_2', 'inv_temp_softmax_3', 'inv_temp_softmax_4', 'lba2', 'lba3', 'lba4', 'lba_3_vs_constraint', 'lba_angle_3', 'lba_angle_3_vs_constraint', 'poisson_race', 'racing_diffusion_3']
engine signatures:
cssm.ddm_flexbound boundary_fun in signature: True
cssm.ddm boundary_fun in signature: False
cssm.addm boundary_fun in signature: False
cssm.lba_vanilla boundary_fun in signature: False
cssm.lba_angle boundary_fun in signature: False
cssm.rlwm_lba_pw_v1 boundary_fun in signature: False
cssm.rlwm_lba_race boundary_fun in signature: False
cssm.poisson_race boundary_fun in signature: True
cssm.racing_diffusion_model boundary_fun in signature: False
lba_angle_3 config: boundary_name='constant' boundary='constant' boundary_params=None
```
(Only the venv path prefix is elided above. `inv_temp_softmax_2/3/4` are choice-only simulators that return `PLACEHOLDER_RT = -1.0` for every trial — `ssms/basic_simulators/inv_temp_softmax.py:10` — so a flat mean RT there is trivially true; that takes the meaningful count from 16 to 13.)
Warning behaviour, checked separately with `warnings.simplefilter("always")` and `n_samples=200`:
```
addm warnings=NONE metadata boundary_fun_type='addm_collapse'
ddm_legacy warnings=NONE metadata boundary_fun_type='constant'
lba2 / lba3 / lba4 warnings=NONE metadata boundary_fun_type=''
lba_3_vs_constraint warnings=NONE metadata boundary_fun_type=''
lba_angle_3 warnings=NONE metadata boundary_fun_type=''
lba_angle_3_vs_constraint warnings=NONE metadata boundary_fun_type=''
dev_rlwm_lba_pw_v1 warnings=NONE metadata boundary_fun_type=''
dev_rlwm_lba_race_v1/v2 warnings=NONE metadata boundary_fun_type=''
poisson_race warnings=["Callable boundary expects parameters ['a'] but none were fou…"]
racing_diffusion_3 warnings=["Callable boundary expects parameters ['a'] but none were fou…"]
```
## Why
Python layer builds and forwards the boundary without checking that the engine accepts one:
- `ssms/basic_simulators/simulator_class.py:388-400` — `_apply_custom_boundary` sets `boundary`/`boundary_name`/`boundary_params` with no engine-capability check.
- `ssms/basic_simulators/simulator_class.py:651-667` — `simulate()` calls `make_boundary_dict` and splats the result into `model_config_local["simulator"](**theta, **boundary_dict, ...)`.
- `ssms/basic_simulators/simulator.py:285-339, 782-788` — same construction and unconditional splat on the functional path.
- `ssms/basic_simulators/simulator_class.py:321-358` — `_infer_simulator_requirements` already computes `supports_boundary = "boundary_fun" in params`, but it is never called anywhere in the package (`grep -rn _infer_simulator_requirements ssms/` returns only the `def` line).
Engines with no `boundary_fun` parameter, so it lands in `**kwargs` and is never read:
- `cssm/ddm_models.pyx:349-363` — `ddm` (model `ddm_legacy`); line 561 hardcodes `boundary_fun_name='constant'` into metadata.
- `cssm/addm_models.pyx:320-338` — `addm`; line 459 hardcodes `boundary_fun_name='addm_collapse'`.
- `cssm/lba_models.pyx:36-46, 144-156, 253-266, 384-396` — `lba_vanilla`, `lba_angle`, `rlwm_lba_pw_v1`, `rlwm_lba_race`. `grep -c boundary_fun cssm/lba_models.pyx` → `0`.
- `cssm/race_models.pyx:917-931` — `racing_diffusion_model`.
- `cssm/poisson_race_models.pyx:59` — declares `boundary_fun = None, # unused, kept for interface compatibility`, so signature introspection gives a false positive here.
Contrast: `cssm/ddm_models.pyx:589-606` (`ddm_flexbound`) declares `boundary_fun`/`boundary_params` and calls `compute_boundary` (`cssm/_utils.pyx:655-666`).
## Suggested fix
Refuse loudly in the Python layer rather than plumbing a boundary the engine cannot use. Wire up the existing dead code in `_apply_custom_boundary` (and the equivalent for `drift`):
```python
reqs = self._infer_simulator_requirements(config["simulator"])
if not reqs["supports_boundary"]:
raise ValueError(
f"Model {config['name']!r} uses simulator "
f"{config['simulator'].__name__!r}, which computes its boundary "
"internally and cannot accept a custom boundary function."
)
```
Signature introspection alone is not enough because of `poisson_race`, so prefer an explicit `"supports_custom_boundary": bool` on each model config (or a module-level set of engines that genuinely honour it), falling back to introspection when absent. The engines to mark `False` are `cssm.ddm`, `cssm.addm`, `cssm.lba_vanilla`, `cssm.lba_angle`, `cssm.rlwm_lba_pw_v1`, `cssm.rlwm_lba_race`, `cssm.racing_diffusion_model`, `cssm.poisson_race` (plus `cssm.full_ddm_hddm_base`, `ddm_models.pyx:55`, which no registry model currently routes to). Put the guard in or immediately after `make_boundary_dict` so `simulator.py:771` is covered too. The sweep above is directly reusable as a parametrised regression test over `get_model_registry().list_models()`: each model must either shorten mean RT measurably under a collapsing bound or raise.
## Is this intended?
Partly, and the issue concedes it: these engines deliberately compute their own boundary (`lba_angle` derives a linear collapse from `theta`; `addm` computes `±(a - b·t)` internally; `poisson_race:59` says so in a comment). The complaint is not that they lack a general boundary hook — it is that the Python layer accepts, validates and forwards a boundary to them with no check, when the check already exists as dead code.
Two things the report explicitly does not claim: the value of `a` is not dropped (`make_boundary_dict` does route it into `boundary_params`, and `a` is also passed independently — the `a=1.5 → 0.3` control moves mean RT 2.863 → 0.379); and "no warning at all" is not universally true, since 5 of the 16 do warn — but incidentally, with the text "expects parameters ['a'] but none were found in theta", which is a missing-parameter complaint, not "this boundary will be ignored".
Unrelated observation while measuring: the LBA engines appear to ignore `random_state` (repeat `simulate(..., random_state=42)` calls give mean RTs differing by ~0.3%). That wobble is why the ignored rows above are not bit-identical; the honouring models move 60-98% over the same comparison, well clear of it.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in ssms/basic_simulators/simulator_class.py at _apply_custom_boundary, _infer_simulator_requirements, and the simulate() boundary splat, then trace the equivalent path in ssms/basic_simulators/simulator.py. Run the supplied model-registry sweep as a regression check. Done means unsupported custom boundaries are rejected on both paths while supported engines continue to honor them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100