Comfy-Org / Comfy-Org/ComfyUI

MiniMaxH3: memory_usage_factor 0.114 underestimates the sampling working set, OOM on 24GB GPUs

Open
#15,781 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Expected Behavior

`MiniMaxH3.memory_usage_factor` should estimate the sampling working set closely
enough that `load_models_gpu()` leaves room for it. The DiT weight budget is
computed as everything above that estimate:

```python
# comfy/model_management.py
minimum_memory_required = max(inference_memory, minimum_memory_required + extra_reserved_memory())
...
lowvram_model_memory = max(0, (current_free_mem - minimum_memory_required),
min(current_free_mem * MIN_WEIGHT_MEMORY_RATIO,
current_free_mem - minimum_inference_memory()))
```

`MIN_WEIGHT_MEMORY_RATIO` is `0.0` on NVIDIA, so the third term is always 0 and
this reduces to **`weight budget = free VRAM - estimated inference memory`**.
There is no safety margin: whatever the estimate is short by gets spent on
resident weights instead.

### Actual Behavior

For MiniMax-H3, `memory_usage_factor = 0.114` (`comfy/supported_models.py`)
underestimates the real sampling working set by roughly **1.45x** on a 24 GB
card, so `load_models_gpu()` pins too much of the 20.97 GB DiT and sampling then
OOMs inside `SamplerCustomAdvanced`.

The margins are tiny, which is why it looks like a flaky/borderline failure
rather than a systematic one. Two real failures:

* 480x720, 107 frames, 20 steps — estimate 2.87 GB, so **20.59 GB of the
20.97 GB model got pinned**, leaving ~3.1 GB for sampling. It OOM'd asking for
**128 MiB**.
* 736x1152, 192 frames, 20 steps — estimate 11.30 GB, 12.22 GB pinned. OOM'd
asking for 1.98 GiB.

Raising the factor to `0.17` (i.e. reserving ~1.49x more) makes both complete.

### Steps to Reproduce

1. Load the MiniMax-H3 DiT (20.97 GB) on a 24 GB GPU, legacy ModelPatcher path
(`--disable-dynamic-vram`; DynamicVRAM does not hit this, see below).
2. Run `MiniMaxH3ImageToVideo` -> `SamplerCustomAdvanced` at, say,
736x1152 with `length=192` and 20 steps.
3. Watch `lowvram_model_memory` come out at ~12.9 GB and the sampler OOM a few
seconds later.

The failure scales with how little the estimate asks for: the *smaller* the job,
the *more* weight gets pinned, so short clips fail just as reliably as long ones.

### Debug Logs

Our build adds `[mem]` logging around `load_models_gpu()`; the values themselves
come straight from the upstream formula quoted above.

480x720 x107 frames — estimate 2.87 GB, 20.59 GB of weights pinned:

```
[mem] load_models_gpu models: [MiniMaxH3], extra_mem: 5.32GB, base_extra_mem: 5.32GB, minimum_memory_required: 2.87GB
[mem] lowvram_budget MiniMaxH3: 20.84GB, free: 23.71GB, cached: 0.00GB, minimum: 2.87GB, model_total: 20.97GB, offloaded: 20.97GB
[mem] model.load MiniMaxH3 partially, total: 20.97GB, loaded: 20.59GB, offloaded: 0.38GB, buffer: 0.27GB, budget: 20.84GB, patches: 0

!!! Exception during processing !!! CUDA out of memory. Tried to allocate 128.00 MiB.
GPU 0 has a total capacity of 23.64 GiB of which 117.25 MiB is free.
Process 5009 has 1.07 GiB memory in use.
Process 79929 has 22.45 GiB memory in use.
Of the allocated memory 21.86 GiB is allocated by PyTorch, and 103.17 MiB is reserved by PyTorch but unallocated.
```

736x1152 x192 frames — estimate 11.30 GB, 12.22 GB of weights pinned:

```
[mem] load_models_gpu models: [MiniMaxH3], extra_mem: 22.18GB, base_extra_mem: 22.18GB, minimum_memory_required: 11.30GB
[mem] lowvram_budget MiniMaxH3: 12.90GB, free: 24.20GB, cached: 0.00GB, minimum: 11.30GB, model_total: 20.97GB, offloaded: 20.97GB
[mem] model.load MiniMaxH3 partially, total: 20.97GB, loaded: 12.22GB, offloaded: 8.74GB, buffer: 0.96GB, budget: 12.90GB, patches: 0

execution_error: node 127 (SamplerCustomAdvanced): CUDA out of memory. Tried to allocate 1.98 GiB.
GPU 0 has a total capacity of 23.52 GiB of which 1.61 GiB is free.
Process 576 has 500.00 MiB memory in use.
Including non-PyTorch memory, this process has 21.39 GiB memory in use.
Of the allocated memory 20.77 GiB is allocated by PyTorch, and 136.61 MiB is reserved by PyTorch but unallocated.
```

(The extra 0.5-1 GiB process in both traces is a second, idle ComfyUI on the same
card holding only its CUDA context. Re-running with that removed changes the
numbers by <1 GiB and does not change any of the outcomes below.)

Same run with the weight budget forced to zero (`--reserve-vram 13`) completes
in 1055 s with a peak of 16133 MiB, i.e. the sampling working set alone is
~15.9 GB against an 11.30 GB estimate — a 1.45x shortfall.

### Other

**Measured results on an RTX 4090 (23.5 GB usable), 20 steps, `res_multistep`:**

| resolution x frames | est. (0.114) | weights pinned | 0.114 | 0.17 |
| --- | --- | --- | --- | --- |
| 480x720 x107 | 2.87 GB | 20.59 -> 19.43 GB | OOM by 128 MiB | pass, 243 s |
| 736x1152 x192 | 11.30 GB | 12.22 -> 6.29 GB | OOM by 1.98 GiB | pass, 1092 s |
| 736x1152 x294 | ~17 GB | 12.90 -> 0 GB | OOM | pass, 1999 s |
| 736x1152 x362 | ~20.9 GB | -> 0 GB | OOM | still OOM by 2.34 GiB |
| 736x1344 x362 | ~24.2 GB | -> 0 GB | OOM | still OOM by 0.45 GiB |

So a higher factor fixes everything up to ~12 s of video on a 24 GB card; the
top two rows genuinely do not fit even with every weight offloaded, and I am not
suggesting the factor can do anything about those.

**The extra reservation is close to free.** Streaming more weights from host RAM
cost under 4% on the same workload and seed: 1055 s with the budget forced to 0
vs 1092 s with 6.29 GB resident. The transfers overlap with compute.

**`--reserve-vram` is not a workaround** for the small-job case: it lowers the
weight budget globally for every model in the process, and small H3 jobs are
exactly the ones where the estimate is proportionally most wrong.

**DynamicVRAM is unaffected.** On a 32 GB card with DynamicVRAM enabled I
replayed a real job at both factors. The computed budget crossed the
full-load/partial-load boundary (23.96 GB -> 19.91 GB, model is 20.97 GB) but
peak VRAM was byte-identical (25849 MiB) and the output differed by one byte.
That matches the code: the DynamicVRAM `ModelPatcher.partially_load()` accepts
`extra_memory` and never uses it.

**Would a different shape of fix be better?** A flat multiplier is the smallest
change, but the underestimate does not look purely multiplicative — fitting the
two measured points suggests a constant term of roughly 0.5-0.6 GB on top. I am
happy to gather more points if that is useful. It may also be worth considering
a small safety margin in the budget formula itself, since today any model whose
`memory_usage_factor` is slightly low turns that directly into an OOM rather
than into a slower run.

**Caveats, in the interest of being upfront:**

* We run a fork. I diffed the two code paths this report depends on against
`master` and both are unmodified: the `minimum_memory_required` computation
and the `lowvram_model_memory` expression are byte-identical.
* The DiT weights are a locally quantized (int8) build of the pruned
`fl2va` release, not a stock file. Inference dtype is still bf16, so the
activation working set should be comparable, but I cannot rule out a
difference.
* Tested both with and without `--use-sage-attention`; the OOM and the fix
behave the same either way (192-frame case: 1086 s without sage vs 1092 s
with, both passing at 0.17).
* torch 2.6.0+cu124, legacy ModelPatcher path.

Happy to run any additional measurement that would help pin down the right
value.

Contributor guide

Open the contributing guide

Research direction

Start in comfy/supported_models.py at MiniMaxH3.memory_usage_factor, then read the minimum_memory_required and lowvram_model_memory calculations in comfy/model_management.py. Reproduce the listed legacy ModelPatcher workflows, compare the sampling memory estimate with the measured working set, and confirm the selected factor avoids OOM for the passing workloads without implying support for jobs that do not fit when weights are offloaded.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.