lllyasviel / lllyasviel/FramePack

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 28.87 GiB. GPU 0 has a total capacity of 8.00 GiB of which 3.68 GiB is free.

Open
#819 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
17.3k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Currently enabled native sdp backends: ['flash', 'math', 'mem_efficient', 'cudnn']
Xformers is not installed!
Flash Attn is not installed!
Sage Attn is not installed!
Namespace(share=False, server='127.0.0.1', port=None, inbrowser=True)
Free VRAM 6.990234375 GB
High-VRAM Mode: False
Downloading shards: 100%|██████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 1002.58it/s]
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 8.18it/s]
Fetching 3 files: 100%|████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 2961.38it/s]
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 26.39it/s]
transformer.high_quality_fp32_output_for_inference = True
* Running on local URL: http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Loaded CLIPTextModel to cuda:0 as complete.
Unloaded CLIPTextModel as complete.
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Loaded SiglipVisionModel to cuda:0 as complete.
latent_padding_size = 27, is_last_section = False
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB
0%| | 0/25 [00:18

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the traceback in webui/diffusers_helper/models/hunyuan_video_packed.py, especially attn_varlen_func, and follow the call path through k_diffusion_hunyuan.py and wrapper.py. Reproduce the CUDA out-of-memory failure using the reported 8 GB GPU configuration; done requires an agreed fix or a clear explanation of the supported memory requirements.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.