lnccbrown / lnccbrown/ssm-simulators
UniformParameterSampler draws parameters in a hash-dependent order, so a seeded sample is not reproducible across processes
- Dominant language
- Jupyter Notebook
- Stars
- 24
- Forks
- 18
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 9
Description
## Describe the bug
`UniformParameterSampler` (via `AbstractParameterSampler._build_dependency_graph` / `_topological_sort`) iterates a `set` when it orders the parameters it draws, so the order in which the RNG is consumed depends on `PYTHONHASHSEED`. With the same `rng` state, two processes can draw different parameter vectors: the samples are only reproducible within one interpreter session.
Found while making LANfactory's derived corpora byte-reproducible (`lanfactory.derive`, unmerged): a corpus file generated from `np.random.default_rng([seed, i])` differed between `PYTHONHASHSEED=0` and `PYTHONHASHSEED=1` runs. LANfactory now draws in `model_config["params"]` order on its side, so it no longer depends on this, but every other consumer of the sampler (the simulation pipeline included) inherits the non-determinism.
## To Reproduce
```bash
for h in 0 1; do PYTHONHASHSEED=$h python - <<'PY'
import numpy as np
from ssms.config import ModelConfigBuilder
from ssms.dataset_generators.parameter_samplers import UniformParameterSampler
cfg = ModelConfigBuilder.from_model("ddm")
s = UniformParameterSampler(param_space=cfg["param_bounds_dict"], constraints=ModelConfigBuilder.get_sampling_transforms(cfg))
out = s.sample(n_samples=3, rng=np.random.default_rng(0))
print({k: np.round(v, 4).tolist() for k, v in out.items()})
PY
done
```
## Expected behavior
The same seed gives the same sample in every process: order the parameters deterministically (e.g. `sorted(...)`, or the config's `params` order) before drawing.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in ssms/dataset_generators/parameter_samplers.py, focusing on AbstractParameterSampler._build_dependency_graph and _topological_sort, then run the provided PYTHONHASHSEED reproduction. Ensure parameter ordering does not depend on set iteration, and verify that identical RNG seeds produce the same sample output across processes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 85/100