lllyasviel / lllyasviel/ControlNet
Integration Challenge: Encoder Architectures in Control Net Input
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
I'm looking to play around with the input for the control net. Basically, I want to hook up some encoder architectures in front of the input into the control net. By that, I mean running the conditions through an encoder before tossing them into the control net as conditions.
My issue is figuring out where to slot in this architecture in the code, especially making sure it gets trained alongside the control net during training.
Thanks a lot in advance for your help — I hope I've explained what I'm aiming for clearly enough.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or training entry point is named. Start by locating the ControlNet input and training pipeline, then clarify which encoder architecture and joint-training behavior are intended. Done would require defined integration and acceptance criteria, which the issue currently does not provide.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100