lllyasviel / lllyasviel/FramePack
It gives an error during generation, the error is related to the memory of the video card
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
In the last steps of generation, when it already seems that everything is completing, it returns an error like this.
I have a video card with 16GB of memory, I tried setting it to all 16GB in the program, I tried 11GB, it gives an error all the time.
can you tell me what the problem might be, and how to fix it so that such errors don't occur?
To create a public link, set share=True in launch().
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Loaded CLIPTextModel to cuda:0 as complete.
Unloaded CLIPTextModel as complete.
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Loaded SiglipVisionModel to cuda:0 as complete.
latent_padding_size = 27, is_last_section = False
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 11.1 GB
0%| | 0/25 [00:03<?, ?it/s]
Traceback (most recent call last):
File "F:\AI_framepack_cu126_torch26\webui\demo_gradio.py", line 241, in worker
generated_latents = sample_hunyuan(
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\pipelines\k_diffusion_hunyuan.py", line 116, in sample_hunyuan
results = sample_unipc(k_model, latents, sigmas, extra_args=sampler_kwargs, disable=False, callback=callback)
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\k_diffusion\uni_pc_fm.py", line 141, in sample_unipc
return FlowMatchUniPC(model, extra_args=extra_args, variant=variant).sample(noise, sigmas=sigmas, callback=callback, disable_pbar=disable)
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\k_diffusion\uni_pc_fm.py", line 118, in sample
model_prev_list = [self.model_fn(x, vec_t)]
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\k_diffusion\uni_pc_fm.py", line 23, in model_fn
return self.model(x, t, **self.extra_args)
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\k_diffusion\wrapper.py", line 37, in k_model
pred_positive = transformer(hidden_states=hidden_states, timestep=timestep, return_dict=False, **extra_args['positive'])[0].float()
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\models\hunyuan_video_packed.py", line 973, in forward
hidden_states, encoder_hidden_states = self.gradient_checkpointing_method(
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\models\hunyuan_video_packed.py", line 832, in gradient_checkpointing_method
result = block(*args)
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\models\hunyuan_video_packed.py", line 652, in forward
attn_output, context_attn_output = self.attn(
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\diffusers\models\attention_processor.py", line 605, in forward
return self.processor(
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\models\hunyuan_video_packed.py", line 151, in call
query = attn.norm_q(query)
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\nn\modules\module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\system\python\lib\site-packages\torch\nn\modules\module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\dit_common.py", line 34, in RMSNorm_forward
variance = hidden_states.to(torch.float32).pow(2).mean(-1, keepdim=True)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 206.00 MiB. GPU 0 has a total capacity of 16.00 GiB of which 10.05 GiB is free. Of the allocated memory 4.68 GiB is allocated by PyTorch, and 91.03 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
there was also such an error at the very beginning and the browser was closed. can you tell me what the problem might be, and how to fix it so that such errors don't occur?
To create a public link, set share=True in launch().
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Loaded CLIPTextModel to cuda:0 as complete.
Unloaded CLIPTextModel as complete.
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Loaded SiglipVisionModel to cuda:0 as complete.
latent_padding_size = 27, is_last_section = False
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB
Traceback (most recent call last):
File "F:\AI_framepack_cu126_torch26\webui\demo_gradio.py", line 216, in worker
move_model_to_device_with_memory_preservation(transformer, target_device=gpu, preserved_memory_gb=gpu_memory_preservation)
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\memory.py", line 92, in move_model_to_device_with_memory_preservation
if hasattr(m, 'weight'):
File "F:\AI_framepack_cu126_torch26\webui\diffusers_helper\memory.py", line 26, in hacked_get_attr
return torch.nn.Parameter(p.to(**kwargs), requires_grad=p.requires_grad)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 72.00 MiB. GPU 0 has a total capacity of 16.00 GiB of which 10.21 GiB is free. Of the allocated memory 4.59 GiB is allocated by PyTorch, and 34.70 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the worker in webui/demo_gradio.py and trace the reported failures through webui/diffusers_helper/memory.py, k_diffusion_hunyuan.py, and models/hunyuan_video_packed.py. Reproduce the CUDA out-of-memory errors using the logged generation path, then verify that the reported generation and model-loading failures are handled or avoided without closing the browser.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100