lllyasviel / lllyasviel/ControlNet
Allow the models to be chained in a sequence. (How to solve the animation issues)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
It's possible to send the result of a model to a secondary and tertiary model pass, where each one has a diferent picture?
That way I could render a picture as a 3D animation in blender and then do img2img and then simply use it initially with the open pose model, then later use a segmentation pass with some mask render from a 3D render using flat unshaded mask animations, then give it later to a model that can take some concept art frame made with a normal text2image prompt and some lora, and use some edge detection model to guide a final polish step.
Maybe as a final step, check between frames using some cache, where each step of the difusion process, is done to all the pictures as a batch system where a frame influences the next frame and maybe some middle value between the previous and next smooth blending operation is applied between the frames.
Then all the animation is done in batch, until all the frames are basically generated in a batch, similar to training with epochs.
That would be my initial idea to solve the animation temporal coherence issue.
Sorry if my english is poor BTW.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or implementation entry points are named. Start by reviewing the proposed model chaining, multi-stage conditioning, and batch animation workflow; this issue would be done only after the project defines a concrete scope and verifies the resulting frames maintain temporal coherence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100