OOM when Minimax H3 VAE encoding a video
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Custom Node Testing
- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
I'm currently using a AMD rx9070 XT gpu, which has 16 GB Vram. I was confronted with OOM when using the r2v workflow with a video input.
### Actual Behavior
OOM because of im2col when computing in Conv. Actually I don't know how to solve it, but I ask claude code to make it computed by fp8 and it finally run successfully.
### Steps to Reproduce
Minimax h3, r2v workflow with a video input.
### Debug Logs
```powershell
[2026-08-05 13:30:36.698] Requested to load MiniMaxH3VideoVAE
[2026-08-05 13:30:37.266] Model MiniMaxH3VideoVAE prepared for dynamic VRAM loading. 4965MB Staged. 0 patches attached. Force pre-loaded 128 weights: 348 KB.
[2026-08-05 13:30:50.977] Model MiniMaxH3VideoVAE prepared for dynamic VRAM loading. 4965MB Staged. 0 patches attached. Force pre-loaded 128 weights: 348 KB.
[2026-08-05 13:30:52.804] Warning: Ran out of memory when regular VAE encoding, retrying with tiled VAE encoding.
[2026-08-05 13:30:54.349] !!! Exception during processing !!! CUDA out of memory. Tried to allocate 7.17 GiB. GPU 0 has a total capacity of 15.92 GiB of which 2.50 GiB is free. Of the allocated memory 6.19 GiB is allocated by PyTorch, and 6.88 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.
[2026-08-05 13:30:54.441] Traceback (most recent call last):
File "/home/user/github/ComfyUI/execution.py", line 545, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(...)
File "/home/user/github/ComfyUI/execution.py", line 344, in get_output_data
return_values = await _async_map_node_over_list(...)
File "/home/user/github/ComfyUI/execution.py", line 318, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "/home/user/github/ComfyUI/execution.py", line 306, in process_inputs
result = f(**inputs)
File "/home/user/github/ComfyUI/comfy_api/internal/__init__.py", line 149, in wrapped_func
return method(locked_class, **inputs)
File "/home/user/github/ComfyUI/comfy_api/latest/_io.py", line 1935, in EXECUTE_NORMALIZED
to_return = cls.execute(*args, **kwargs)
File "/home/user/github/ComfyUI/comfy_extras/nodes_minimax_h3.py", line 254, in execute
z = vae.encode(frames)
File "/home/user/github/ComfyUI/comfy/sd.py", line 1297, in encode
samples = self._encode_tiled_owned(pixel_samples, tile_x=tile, tile_y=tile, overlap=overlap)
File "/home/user/github/ComfyUI/comfy/sd.py", line 1138, in _encode_tiled_owned
out = self.first_stage_model.encode_tiled(x, **kwargs)
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 675, in encode_tiled
return self.encode(x)
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 665, in encode
moments = self.encode_temporal(x)
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 533, in encode_temporal
z_list.append(self._adaptive_encode(clip_x))
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 392, in _adaptive_encode
return self.tiled_encode(x)
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 461, in tiled_encode
row.append(self._encode_moments(tile))
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 385, in _encode_moments
return self.quant_conv(self.encoder(x))
...
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 159, in forward
h = self.down[i_level].block[i_block](h)
...
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 106, in forward
h = self.conv1(F.silu(self.norm1(x), inplace=True))
...
File "/home/user/github/ComfyUI/comfy/ldm/minimax/vae.py", line 55, in forward
return super().forward(x)
File "/home/user/github/ComfyUI/comfy/ops.py", line 609, in forward
return self.forward_comfy_cast_weights(*args, **kwargs)
File "/home/user/github/ComfyUI/comfy/ops.py", line 602, in forward_comfy_cast_weights
x = self._conv_forward(input, weight, bias, autopad=autopad)
File "/home/user/github/ComfyUI/comfy/ops.py", line 598, in _conv_forward
return super()._conv_forward(input, weight, bias, *args, **kwargs)
File ".../torch/nn/modules/conv.py", line 730, in _conv_forward
return F.conv3d(
input, weight, bias, self.stride, self.padding, self.dilation, self.groups
)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 7.17 GiB. GPU 0 has a total capacity of 15.92 GiB of which 2.50 GiB is free. Of the allocated memory 6.19 GiB is allocated by PyTorch, and 6.88 GiB is reserved by PyTorch but unallocated.
```
### Other
_No response_
Contributor guide
Research direction
Reproduce the Minimax H3 r2v workflow with video input on the reported 16 GB GPU, then trace the failure from comfy_extras/nodes_minimax_h3.py through comfy/ldm/minimax/vae.py and the convolution path in comfy/ops.py. Use the supplied traceback to identify the memory-heavy encoding step; done means the workflow completes without the reported out-of-memory failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100