modelscope / modelscope/DiffSynth-Studio
Silently slower attention kernel when training MiniMax-H3 LoRA on B200
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
I ran the repo's MiniMax-H3 LoRA training example (examples/minimax_h3/model_training/lora/MiniMax-H3-FL2VA.sh, stage 2) on 8x B200, following the installation README, and got 6.32 s per training step. A profile showed attention running on the FA2 kernel, which is an sm80-era design and much slower on this GPU than torch SDPA's cuDNN backend. With the kernel switched, the same step took 3.80 s. Nothing in the logs said which implementation had been picked, so I only found it by profiling.
Setting DIFFSYNTH_ATTENTION_IMPLEMENTATION=torch avoids this, as @mi804 noted in #1680. A line in the install docs, or a one-line log of the chosen implementation at import, would have saved me the profile.
I opened #1680 to record my experiment: setup, measurements, traces and some potential fixes. A different fix may also well suit the codebase.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/minimax_h3/model_training/lora/MiniMax-H3-FL2VA.sh at stage 2, the installation README, and the experiment recorded in #1680. Reproduce the attention implementation choice with DIFFSYNTH_ATTENTION_IMPLEMENTATION=torch and inspect the existing logging or selection path. Done means the selected implementation is visible to users or the installation guidance clearly prevents the silent slowdown.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100