aigc-apps / aigc-apps/VideoX-Fun
如何开启 Sequence Parallel 训练
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 188
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 3
Description
对于 14B 的模型如果分辨率较大，序列长度较长，单存 FSDP 训练还是会现存不够，怎么开启 Sequence Parallel ，或者其他节省现存的方式训练？
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named. Start by locating the FSDP training configuration and checking whether Sequence Parallel is supported for long sequences and high-resolution 14B training. Done means documenting a verified setup or clearly supported alternatives for reducing memory use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100