DLR-RM / DLR-RM/stable-baselines3

[Bug]: VecEnv sub-environment seeds (seed + i) overlap across runs with adjacent base seeds

Open
#2,268 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
13.8k
Forks
2.2k
Avg merge
1h 35m
Merged PRs (30d)
2

Description

### 🐛 Bug

`VecEnv.seed(seed)` (and `make_vec_env(..., seed=seed)`) seeds sub-environment `i` with `seed + i`, which causes issues with the standard workflow of sweeping seeds `0..N` to get independent runs for evaluation.

As a result of the sub-environment seeding, two runs with adjacent base seeds share most of their sub-environment seeds: with `n_envs=4`, a run with `seed=0` uses env seeds `{0,1,2,3}` and a run with `seed=1` uses `{1,2,3,4}`, i.e., 3 of 4 identical. Since env seeding is deterministic, the shared sub-envs produce identical RNG streams across the two runs. The runs may still end up partially differing due to other sources of randomness (e.g., due to input actions that may differ across sub-environments despite their identical seed); however the problem remains valid, and the issue in particular remains the same if the same root seed is also used indirectly to determine the actions.

A simple fix is to derive sub-env seeds via `np.random.SeedSequence(seed).spawn(n_envs)` (or `generate_state(n_envs)`) instead of `seed + i`, so any two distinct base seeds yield disjoint, independent env streams.

### To Reproduce

```python
import numpy as np

from stable_baselines3.common.env_util import make_vec_env

venv_a = make_vec_env("Pendulum-v1", n_envs=4, seed=0)
venv_b = make_vec_env("Pendulum-v1", n_envs=4, seed=1)

obs_a = venv_a.reset()
obs_b = venv_b.reset()

print("run A (seed=0) sub-env seeds:", venv_a.seed(0)) # [0, 1, 2, 3]
print("run B (seed=1) sub-env seeds:", venv_b.seed(1)) # [1, 2, 3, 4]
print("A[1:] == B[:3]:", np.allclose(obs_a[1:], obs_b[:3])) # True
```

### Relevant log output / Error message

```shell
run A (seed=0) sub-env seeds: [0, 1, 2, 3]
run B (seed=1) sub-env seeds: [1, 2, 3, 4]
A[1:] == B[:3]: True
```

### System Info

- OS: Linux-7.0.0-27-generic-x86_64-with-glibc2.43 # 27-Ubuntu SMP PREEMPT_DYNAMIC Thu Jun 18 19:13:49 UTC 2026
- Python: 3.14.4
- Stable-Baselines3: 2.8.0a4
- PyTorch: 2.13.0+cu130
- GPU Enabled: True
- Numpy: 2.5.1
- Cloudpickle: 3.1.2
- Gymnasium: 1.2.3

### Checklist

- [x] My issue does not relate to a custom gym environment. (Use the custom gym env template instead)
- [x] I have checked that there is no similar [issue](https://github.com/DLR-RM/stable-baselines3/issues) in the repo
- [x] I have read the [documentation](https://stable-baselines3.readthedocs.io/en/master/)
- [x] I have provided a [minimal and working](https://github.com/DLR-RM/stable-baselines3/issues/982#issuecomment-1197044014) example to reproduce the bug
- [x] I've used the [markdown code blocks](https://help.github.com/en/articles/creating-and-highlighting-code-blocks) for both code and stack traces.

Contributor guide

Open the contributing guide

Research direction

Start at VecEnv.seed and make_vec_env, then run the provided Pendulum-v1 reproduction to confirm the overlapping sub-environment streams. Replace the adjacent-seed behavior with independent derivation for each sub-environment and verify that distinct base seeds no longer produce matching sub-environment observations or seed lists.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.