lllyasviel / lllyasviel/ControlNet

Question about speed and memory consumption

Open
#115 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
34.1k
Forks
3k
PR merge metrics
No merged PRs in 30d

Description

This paragraph from the paper confuses me:

`As shown in Fig. 3, we use ControlNet to control each level of the U-net. Note that the way we connect the ControlNet is computationally efficient: since the original weights are locked, no gradient computation on the original encoder is needed for training. This can speed up training and save GPU memory, as half of the gradient computation on the original model can be avoided. Training a stable diffusion model with ControlNet requires only about 23% more GPU memory and 34% more time in each training iteration (as tested on a single Nvidia A100 PCIE 40G).`

The first half claims memory savings and speed gain while the part about SD + CN shows the opposite.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the quoted paper paragraph and comparing its claims about ControlNet, the U-net, stable diffusion, GPU memory, and training time. The issue names no repository files, tests, or entry points; done means providing a clear explanation that reconciles the claimed savings with the reported additional resource use.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.