res_multistep allocates and discards a full-size noise tensor every step (eta is always 0)
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
## Expected Behavior
`res_multistep` and `res_multistep_cfg_pp` run with `eta=0`, so no noise is injected. I'd
expect them not to allocate a noise tensor at all — the same way every other sampler in
`sampling.py` that exposes an `eta=0` wrapper already skips it.
## Actual Behavior
They allocate a full latent-sized `randn` on device every step, multiply it by zero, and
throw it away.
The noise injection in `res_multistep` is guarded on `sigmas[i + 1] > 0` alone:
https://github.com/Comfy-Org/ComfyUI/blob/16e3f303/comfy/k_diffusion/sampling.py#L1447-L1449
```python
# Noise addition
if sigmas[i + 1] > 0:
x = x + noise_sampler(sigmas[i], sigmas[i + 1]) * s_noise * sigma_up
```
But both public wrappers hard-code `eta=0.`:
```python
def sample_res_multistep(model, x, sigmas, ...): # L1459
return res_multistep(..., eta=0., cfg_pp=False)
def sample_res_multistep_cfg_pp(model, x, sigmas, ...): # L1462
return res_multistep(..., eta=0., cfg_pp=True)
```
and with `eta` falsy, `get_ancestral_step` returns early with `sigma_up = 0.` (L71):
```python
if not eta:
return sigma_to, 0.
```
So on every step except the last, `noise_sampler(...)` allocates a tensor the size of the
latent, `* s_noise` and `* sigma_up` produce two more, and the result added to `x` is
zeros. The generic `res_multistep(..., eta=1.)` signature is fine — only the two
zero-`eta` entry points are affected, and those are the ones users actually select.
**This appears to be an oversight rather than intent, because the rest of the file already
guards this exact pattern:**
- L780, L790, L868, L936, L1305 — `if eta > 0 and s_noise > 0:`
- L1606, L1678 — `inject_noise = eta > 0 and s_noise > 0`
- L731 — `if sigmas[i + 1] > 0 and eta > 0:`
The closest analogue is exact: `sample_euler_cfg_pp` (L1311) delegates with
`eta=0.0, s_noise=0.0` into a body that *is* guarded at L1305, and `sample_seeds_2_cfg_pp`
(L1657) does the same into the L1606 guard. `res_multistep` is the only one missing it.
For completeness, the other unguarded `noise_sampler` call sites (L236, L368, L681, L1357)
are all inside `*_ancestral*` functions that default to `eta=1.`, where the noise is real
and the current code is correct.
### Suggested fix
One line, matching the style used at L780/L790/L868/L936/L1305:
```diff
- if sigmas[i + 1] > 0:
+ if sigmas[i + 1] > 0 and eta > 0 and s_noise > 0:
x = x + noise_sampler(sigmas[i], sigmas[i + 1]) * s_noise * sigma_up
```
**This does not change output or break seed reproducibility.** `noise_sampler` is the only
consumer of that generator anywhere in `res_multistep`, so skipping the call doesn't
desync the RNG stream for later steps — and the term it feeds is multiplied by `0.` today
regardless. Existing seeds reproduce exactly.
### Why it matters
On image latents this is just wasted bandwidth. On video latents it's the difference
between fitting and OOMing: the discarded tensors are the same size as the working latent,
and they land at the peak of the step when the model's activations are already resident.
`res_multistep` is not an obscure choice here — it's what Comfy-Org's own MiniMax H3
templates ship with in `KSamplerSelect`.
## Steps to Reproduce
1. Load the official **MiniMax H3 ref2va** template (ships with `res_multistep`).
2. Set resolution to 0.3 MP, duration 15 s (→ 362 frames via the template's
`ComfyMathExpression` node), 20 steps.
3. Run on an 8 GB card.
It OOMs at `sampling.py:1449`. The same workflow at **0.2 MP / 20 s (481 frames) completes
fine** — despite having 33% *more* frames, its total latent volume is ~11% smaller
(`0.2 × 481 ≈ 96` vs `0.3 × 362 ≈ 109`). So the config that fails is only ~13% over one
that works, a margin comfortably inside what these discarded allocations account for.
## Debug Logs
```
File "/home/paul/ComfyUI/comfy/k_diffusion/sampling.py", line 1460, in sample_res_multistep
return res_multistep(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, s_noise=s_noise, noise_sampler=noise_sampler, eta=0., cfg_pp=False)
File "/home/paul/ComfyUI/comfy/k_diffusion/sampling.py", line 1449, in res_multistep
x = x + noise_sampler(sigmas[i], sigmas[i + 1]) * s_noise * sigma_up
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/paul/ComfyUI/comfy/k_diffusion/sampling.py", line 88, in
return lambda sigma, sigma_next: torch.randn(x.size(), dtype=x.dtype, layout=x.layout, device=x.device, generator=generator)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: out of memory
```
Torch's own allocator reported only 136 MB reserved at the time of the OOM — the request
that failed was this noise tensor, not model weights.
## Other
- **Version:** `16e3f303` (master)
- **Environment:** Linux, RTX 5050 8 GB, driver 610.57.04, torch 2.11.0+cu130, Python 3.14
- Custom nodes were not disabled for this run, but the finding is a static read of
`sampling.py` on master and doesn't depend on them.
Contributor guide
Research direction
Start in comfy/k_diffusion/sampling.py at res_multistep around lines 1447-1449, then compare the existing eta and s_noise guards at the other call sites listed in the issue. Reproduce the MiniMax H3 workflow or inspect the sampler path to confirm that zero-eta entry points do not call noise_sampler, while generic eta behavior and output remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100