lllyasviel / lllyasviel/FramePack
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 29.76 GiB
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
RTX 2060 Super 8Gb INNO3D TWIN X2
So I extended Virtual Memory to 51200MB
With Flash Attn -- "flashAttention only supports Ampere GPUs or newer", so I uninstalled 'Flash Attn' and run without anything
```
Currently enabled native sdp backends: ['flash', 'math', 'mem_efficient', 'cudnn']
Xformers is not installed!
Flash Attn is not installed!
Sage Attn is not installed!
Namespace(share=False, server='127.0.0.1', port=None, inbrowser=True)
Free VRAM 6.984375 GB
High-VRAM Mode: False
Downloading shards: 100%|██████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 3982.25it/s]
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 5.96it/s]
Fetching 3 files: 100%|████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 2994.51it/s]
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 6.68it/s]
transformer.high_quality_fp32_output_for_inference = True
* Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Loaded CLIPTextModel to cuda:0 as complete.
Unloaded CLIPTextModel as complete.
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Loaded SiglipVisionModel to cuda:0 as complete.
latent_padding_size = 27, is_last_section = False
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB
0%| | 0/15 [00:13
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at webui/demo_gradio.py's worker and trace sample_hunyuan into webui/diffusers_helper/models/hunyuan_video_packed.py, especially attn_varlen_func. Reproduce the reported generation on the RTX 2060 Super and inspect the failing scaled_dot_product_attention call. Done means the reported workload no longer raises this CUDA out-of-memory error, with the relevant memory behavior verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100