huggingface / huggingface/diffusers

High memory consumption for HunyuanVideo on CPU

Open
#10,569 2 comments 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### Describe the bug

We have a 4th-gen Xeon Scalable system that we are trying to run HunyuanVideo (via Diffusers) on. Remarkably, the demo code runs out of the box with no tweaks, which is a testament to the quality of the Intel PyTorch code :) However, during inference we see very high memory usage - over 180GB - to generate a 512x320x129 frame video; this is 3x what the CUDA implementation needs to generate a 1280x720x129 frame video. My suspicion is that this is because PyTorch SDPA doesn't have a memory efficient implementation on CPU.

### Reproduction

```
import torch
from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel
from diffusers.utils import export_to_video

model_id = "hunyuanvideo-community/HunyuanVideo"
transformer = HunyuanVideoTransformer3DModel.from_pretrained(
model_id, subfolder="transformer", torch_dtype=torch.bfloat16
)

pipe = HunyuanVideoPipeline.from_pretrained(model_id, transformer=transformer, torch_dtype=torch.float16)
pipe.vae.enable_tiling()

with torch.no_grad():
output = pipe(
prompt="A walk through tokyo at night, realistic",
height=320,
width=512,
num_frames=129,
num_inference_steps=50,
).frames[0]
export_to_video(output, "output4.mp4", fps=24)
```

### System Info

- 🤗 Diffusers version: 0.33.0.dev0
- Platform: Linux-6.5.0-41-generic-x86_64-with-glibc2.35
- Running on Google Colab?: No
- Python version: 3.10.12
- PyTorch version (GPU?): 2.6.0.dev20241225+cpu (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.27.0
- Transformers version: 4.46.3
- Accelerate version: 1.2.1
- PEFT version: 0.6.0
- Bitsandbytes version: not installed
- Safetensors version: 0.4.5
- xFormers version: not installed
- Accelerator: NA
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@DN6 @a-r-r-o-w

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.