modelscope / modelscope/DiffSynth-Studio
qwen-image-edit lora训练 是否有显存优化手段
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
使用单机8卡49GA6000，按照repo中的lora去训练，已经启动了 --use_gradient_checkpointing --use_gradient_checkpointing_offload --enable_fp8_training，降低了lora_rank，zero_stage2/3都试过，全都在accelerator.prepare步骤爆显存
求各位指导：
- 是否是deepspeed配置没起作用？配置方式见下图（--config_file）
- 是否有其他方法能够优化显存？
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The report concerns LoRA training for qwen-image-edit and an out-of-memory failure at accelerator.prepare, despite the listed gradient-checkpointing, FP8, LoRA rank, and ZeRO settings. Start by reproducing the repository's documented LoRA training command and checking whether the supplied --config_file is applied. Done would require identifying a confirmed configuration or training change that prevents the reported memory failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100