huggingface / huggingface/diffusers
Finalize Guidelines for Ulysses based context parallel attention
Open
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
The PR https://github.com/huggingface/diffusers/pull/13924 added support for Ulysses Context parallel attention but looks like Library wide support for such an attention pattern still needs to be decided.
Contributor guide
Research direction
Start by reviewing PR #13924, which added Ulysses Context Parallel attention support. The issue names no files, tests, or entry points, so first determine the library-wide scope and the decisions still needed for this attention pattern. Done means the support guidelines and their intended application across the library are agreed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100