DLR-RM / DLR-RM/stable-baselines3
[Bug]: VecEnv sub-environment seeds (seed + i) overlap across runs with adjacent base seeds
- Dominant language
- Python
- Stars
- 13.8k
- Forks
- 2.2k
- Avg merge
- 1h 35m
- Merged PRs (30d)
- 2
Description
### 🐛 Bug
`VecEnv.seed(seed)` (and `make_vec_env(..., seed=seed)`) seeds sub-environment `i` with `seed + i`, which causes issues with the standard workflow of sweeping seeds `0..N` to get independent runs for evaluation.
As a result of the sub-environment seeding, two runs with adjacent base seeds share most of their sub-environment seeds: with `n_envs=4`, a run with `seed=0` uses env seeds `{0,1,2,3}` and a run with `seed=1` uses `{1,2,3,4}`, i.e., 3 of 4 identical. Since env seeding is deterministic, the shared sub-envs produce identical RNG streams across the two runs. The runs may still end up partially differing due to other sources of randomness (e.g., due to input actions that may differ across sub-environments despite their identical seed); however the problem remains valid, and the issue in particular remains the same if the same root seed is also used indirectly to determine the actions.
A simple fix is to derive sub-env seeds via `np.random.SeedSequence(seed).spawn(n_envs)` (or `generate_state(n_envs)`) instead of `seed + i`, so any two distinct base seeds yield disjoint, independent env streams.
### To Reproduce
```python
import numpy as np
from stable_baselines3.common.env_util import make_vec_env
venv_a = make_vec_env("Pendulum-v1", n_envs=4, seed=0)
venv_b = make_vec_env("Pendulum-v1", n_envs=4, seed=1)
obs_a = venv_a.reset()
obs_b = venv_b.reset()
print("run A (seed=0) sub-env seeds:", venv_a.seed(0)) # [0, 1, 2, 3]
print("run B (seed=1) sub-env seeds:", venv_b.seed(1)) # [1, 2, 3, 4]
print("A[1:] == B[:3]:", np.allclose(obs_a[1:], obs_b[:3])) # True
```
### Relevant log output / Error message
```shell
run A (seed=0) sub-env seeds: [0, 1, 2, 3]
run B (seed=1) sub-env seeds: [1, 2, 3, 4]
A[1:] == B[:3]: True
```
### System Info
- OS: Linux-7.0.0-27-generic-x86_64-with-glibc2.43 # 27-Ubuntu SMP PREEMPT_DYNAMIC Thu Jun 18 19:13:49 UTC 2026
- Python: 3.14.4
- Stable-Baselines3: 2.8.0a4
- PyTorch: 2.13.0+cu130
- GPU Enabled: True
- Numpy: 2.5.1
- Cloudpickle: 3.1.2
- Gymnasium: 1.2.3
### Checklist
- [x] My issue does not relate to a custom gym environment. (Use the custom gym env template instead)
- [x] I have checked that there is no similar [issue](https://github.com/DLR-RM/stable-baselines3/issues) in the repo
- [x] I have read the [documentation](https://stable-baselines3.readthedocs.io/en/master/)
- [x] I have provided a [minimal and working](https://github.com/DLR-RM/stable-baselines3/issues/982#issuecomment-1197044014) example to reproduce the bug
- [x] I've used the [markdown code blocks](https://help.github.com/en/articles/creating-and-highlighting-code-blocks) for both code and stack traces.
Contributor guide
Research direction
Start at VecEnv.seed and make_vec_env, then run the provided Pendulum-v1 reproduction to confirm the overlapping sub-environment streams. Replace the adjacent-seed behavior with independent derivation for each sub-environment and verify that distinct base seeds no longer produce matching sub-environment observations or seed lists.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100