lllyasviel / lllyasviel/FramePack

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 108.00 MiB. GPU 0 has a total capacity of 24.00 GiB of which 15.34 GiB is free.

Open
#132 14 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.3k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

PS D:\framepack_cu126_torch26> .\run.bat
Currently enabled native sdp backends: ['flash', 'math', 'mem_efficient', 'cudnn']
Xformers is not installed!
Flash Attn is not installed!
Sage Attn is not installed!
Namespace(share=False, server='127.0.0.1', port=None, inbrowser=True)
Free VRAM 22.759765625 GB
High-VRAM Mode: False
Downloading shards: 100%|███████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 797.78it/s]
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 5.46it/s]
Fetching 3 files: 100%|████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 3118.44it/s]
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 12.71it/s]
transformer.high_quality_fp32_output_for_inference = True

To create a public link, set share=True in launch().
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Loaded CLIPTextModel to cuda:0 as complete.
Unloaded CLIPTextModel as complete.
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Loaded SiglipVisionModel to cuda:0 as complete.
latent_padding_size = 27, is_last_section = False
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB
Traceback (most recent call last):
File "D:\framepack_cu126_torch26\webui\demo_gradio.py", line 216, in worker
move_model_to_device_with_memory_preservation(transformer, target_device=gpu, preserved_memory_gb=gpu_memory_preservation)
File "D:\framepack_cu126_torch26\webui\diffusers_helper\memory.py", line 92, in move_model_to_device_with_memory_preservation
if hasattr(m, 'weight'):
File "D:\framepack_cu126_torch26\webui\diffusers_helper\memory.py", line 26, in hacked_get_attr
return torch.nn.Parameter(p.to(**kwargs), requires_grad=p.requires_grad)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 108.00 MiB. GPU 0 has a total capacity of 24.00 GiB of which 15.34 GiB is free. Of the allocated memory 7.33 GiB is allocated by PyTorch, and 33.95 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Press any key to continue . . .

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with run.bat and read webui/demo_gradio.py around worker line 216 and webui/diffusers_helper/memory.py around lines 26 and 92. Investigate the reported CUDA out-of-memory failure while moving the transformer to cuda:0, then verify the same setup can complete that step without the error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.