Comfy-Org / Comfy-Org/ComfyUI

res_multistep allocates and discards a full-size noise tensor every step (eta is always 0)

Open Beginner friendly
#15,328 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

## Expected Behavior

`res_multistep` and `res_multistep_cfg_pp` run with `eta=0`, so no noise is injected. I'd
expect them not to allocate a noise tensor at all — the same way every other sampler in
`sampling.py` that exposes an `eta=0` wrapper already skips it.

## Actual Behavior

They allocate a full latent-sized `randn` on device every step, multiply it by zero, and
throw it away.

The noise injection in `res_multistep` is guarded on `sigmas[i + 1] > 0` alone:

https://github.com/Comfy-Org/ComfyUI/blob/16e3f303/comfy/k_diffusion/sampling.py#L1447-L1449

```python
# Noise addition
if sigmas[i + 1] > 0:
x = x + noise_sampler(sigmas[i], sigmas[i + 1]) * s_noise * sigma_up
```

But both public wrappers hard-code `eta=0.`:

```python
def sample_res_multistep(model, x, sigmas, ...): # L1459
return res_multistep(..., eta=0., cfg_pp=False)

def sample_res_multistep_cfg_pp(model, x, sigmas, ...): # L1462
return res_multistep(..., eta=0., cfg_pp=True)
```

and with `eta` falsy, `get_ancestral_step` returns early with `sigma_up = 0.` (L71):

```python
if not eta:
return sigma_to, 0.
```

So on every step except the last, `noise_sampler(...)` allocates a tensor the size of the
latent, `* s_noise` and `* sigma_up` produce two more, and the result added to `x` is
zeros. The generic `res_multistep(..., eta=1.)` signature is fine — only the two
zero-`eta` entry points are affected, and those are the ones users actually select.

**This appears to be an oversight rather than intent, because the rest of the file already
guards this exact pattern:**

- L780, L790, L868, L936, L1305 — `if eta > 0 and s_noise > 0:`
- L1606, L1678 — `inject_noise = eta > 0 and s_noise > 0`
- L731 — `if sigmas[i + 1] > 0 and eta > 0:`

The closest analogue is exact: `sample_euler_cfg_pp` (L1311) delegates with
`eta=0.0, s_noise=0.0` into a body that *is* guarded at L1305, and `sample_seeds_2_cfg_pp`
(L1657) does the same into the L1606 guard. `res_multistep` is the only one missing it.

For completeness, the other unguarded `noise_sampler` call sites (L236, L368, L681, L1357)
are all inside `*_ancestral*` functions that default to `eta=1.`, where the noise is real
and the current code is correct.

### Suggested fix

One line, matching the style used at L780/L790/L868/L936/L1305:

```diff
- if sigmas[i + 1] > 0:
+ if sigmas[i + 1] > 0 and eta > 0 and s_noise > 0:
x = x + noise_sampler(sigmas[i], sigmas[i + 1]) * s_noise * sigma_up
```

**This does not change output or break seed reproducibility.** `noise_sampler` is the only
consumer of that generator anywhere in `res_multistep`, so skipping the call doesn't
desync the RNG stream for later steps — and the term it feeds is multiplied by `0.` today
regardless. Existing seeds reproduce exactly.

### Why it matters

On image latents this is just wasted bandwidth. On video latents it's the difference
between fitting and OOMing: the discarded tensors are the same size as the working latent,
and they land at the peak of the step when the model's activations are already resident.

`res_multistep` is not an obscure choice here — it's what Comfy-Org's own MiniMax H3
templates ship with in `KSamplerSelect`.

## Steps to Reproduce

1. Load the official **MiniMax H3 ref2va** template (ships with `res_multistep`).
2. Set resolution to 0.3 MP, duration 15 s (→ 362 frames via the template's
`ComfyMathExpression` node), 20 steps.
3. Run on an 8 GB card.

It OOMs at `sampling.py:1449`. The same workflow at **0.2 MP / 20 s (481 frames) completes
fine** — despite having 33% *more* frames, its total latent volume is ~11% smaller
(`0.2 × 481 ≈ 96` vs `0.3 × 362 ≈ 109`). So the config that fails is only ~13% over one
that works, a margin comfortably inside what these discarded allocations account for.

## Debug Logs

```
File "/home/paul/ComfyUI/comfy/k_diffusion/sampling.py", line 1460, in sample_res_multistep
return res_multistep(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, s_noise=s_noise, noise_sampler=noise_sampler, eta=0., cfg_pp=False)
File "/home/paul/ComfyUI/comfy/k_diffusion/sampling.py", line 1449, in res_multistep
x = x + noise_sampler(sigmas[i], sigmas[i + 1]) * s_noise * sigma_up
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/paul/ComfyUI/comfy/k_diffusion/sampling.py", line 88, in
return lambda sigma, sigma_next: torch.randn(x.size(), dtype=x.dtype, layout=x.layout, device=x.device, generator=generator)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.AcceleratorError: CUDA error: out of memory
```

Torch's own allocator reported only 136 MB reserved at the time of the OOM — the request
that failed was this noise tensor, not model weights.

## Other

- **Version:** `16e3f303` (master)
- **Environment:** Linux, RTX 5050 8 GB, driver 610.57.04, torch 2.11.0+cu130, Python 3.14
- Custom nodes were not disabled for this run, but the finding is a static read of
`sampling.py` on master and doesn't depend on them.

Contributor guide

Open the contributing guide

Research direction

Start in comfy/k_diffusion/sampling.py at res_multistep around lines 1447-1449, then compare the existing eta and s_noise guards at the other call sites listed in the issue. Reproduce the MiniMax H3 workflow or inspect the sampler path to confirm that zero-eta entry points do not call noise_sampler, while generic eta behavior and output remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.