modelscope / modelscope/DiffSynth-Studio
Why there is no unconditional input while we training the model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
Hi @lzws Thanks for your excellent work for the AI open-sourcing development.
I have a question about the model training pipeline. From the code
https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/wanvideo/model_training/train.py#L58, there is no unconditional input while training the model. But, when it do inference, the cfg_scale=5, it means that the model needs the unconditional input.
From the CFG paper (https://arxiv.org/pdf/2207.12598), when we set cfg_scale != 1, we should add probability to input the unconditional parameters when we training the model.
So, could we input the unconditional parameters while we training the model?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/wanvideo/model_training/train.py around line 58 and compare its training inputs with the inference path using cfg_scale=5. Read the linked CFG paper to verify the expected conditioning behavior; completion requires a maintainer decision on whether unconditional inputs belong in training and an agreed validation case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100