lllyasviel / lllyasviel/ControlNet
Question about speed and memory consumption
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
This paragraph from the paper confuses me:
`As shown in Fig. 3, we use ControlNet to control each level of the U-net. Note that the way we connect the ControlNet is computationally efficient: since the original weights are locked, no gradient computation on the original encoder is needed for training. This can speed up training and save GPU memory, as half of the gradient computation on the original model can be avoided. Training a stable diffusion model with ControlNet requires only about 23% more GPU memory and 34% more time in each training iteration (as tested on a single Nvidia A100 PCIE 40G).`
The first half claims memory savings and speed gain while the part about SD + CN shows the opposite.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the quoted paper paragraph and comparing its claims about ControlNet, the U-net, stable diffusion, GPU memory, and training time. The issue names no repository files, tests, or entry points; done means providing a clear explanation that reconciles the claimed savings with the reported additional resource use.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100