huggingface / huggingface/diffusers

[feature] Add context parallel cp_plan support to many mainstream models

Open
#12,603 4 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

I am working in cache-dit, where I use diffusers as the **native** context parallel backend and extensively utilize custom cp_plan to support context parallel for mainstream models (and has also make many models worked with context parallel + attn_mask via native attn backend). These models include: FLUX, Qwen-Image, LTXVideo, Wan, HunyuanImage, HunyuanVideo, CogVideoX, CogView3Plus, CogView4, Qwen-Image-Lightning, ConsisID, Chroma, VisualCloze, etc. If you wish to integrate these implementations into diffusers, I am willing to submit some PRs to provide support. I will keep a long-term focus on the context parallel feature of diffusers and hope it can become an extremely user-friendly function. our experimental cp_planners are at: [cp_planners](https://github.com/vipshop/cache-dit/tree/main/src/cache_dit/parallelism/backends/native_diffusers/context_parallelism)

Image

@sayakpaul @yiyixuxu @DN6

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the experimental cp_planners in cache-dit/src/cache_dit/parallelism/backends/native_diffusers/context_parallelism and compare them with diffusers' existing context-parallel support. The issue lists FLUX, Qwen-Image, LTXVideo, Wan, HunyuanImage, HunyuanVideo, and other models, but names no diffusers files or tests. Done would require a scoped integration plan and support for the selected mainstream models.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
distributed-systems, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.