huggingface / huggingface/diffusers
[feature] Add context parallel cp_plan support to many mainstream models
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
I am working in cache-dit, where I use diffusers as the **native** context parallel backend and extensively utilize custom cp_plan to support context parallel for mainstream models (and has also make many models worked with context parallel + attn_mask via native attn backend). These models include: FLUX, Qwen-Image, LTXVideo, Wan, HunyuanImage, HunyuanVideo, CogVideoX, CogView3Plus, CogView4, Qwen-Image-Lightning, ConsisID, Chroma, VisualCloze, etc. If you wish to integrate these implementations into diffusers, I am willing to submit some PRs to provide support. I will keep a long-term focus on the context parallel feature of diffusers and hope it can become an extremely user-friendly function. our experimental cp_planners are at: [cp_planners](https://github.com/vipshop/cache-dit/tree/main/src/cache_dit/parallelism/backends/native_diffusers/context_parallelism)
@sayakpaul @yiyixuxu @DN6
Contributor guide
Research direction
Start by reviewing the experimental cp_planners in cache-dit/src/cache_dit/parallelism/backends/native_diffusers/context_parallelism and compare them with diffusers' existing context-parallel support. The issue lists FLUX, Qwen-Image, LTXVideo, Wan, HunyuanImage, HunyuanVideo, and other models, but names no diffusers files or tests. Done would require a scoped integration plan and support for the selected mainstream models.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100