invoke-ai / invoke-ai/InvokeAI
[enhancement]: Concat ControlNet(s) for easier multiple control nets into a denoise latent
- Dominant language
- Python
- Stars
- 28.2k
- Forks
- 3k
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 19
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Contact Details
_No response_
### What should this feature add?
In workflows, the `DenoiseLatents` steps takes a ControlNet or a List[ControlNet], but the only way to get one (that I found) is via a collect -- this doesn't work when you're inside of an iteration, as it collects EVERY control net before passing them along. You can have an iterated control net off of, say, [a video](https://github.com/helix4u/load_video_frame), and then have a fixed set of control nets to keep the context like a background or color palette.
Here's a more or less generic `Concat` node that does `concat(T|T[], T|T[]): T[]` (similar to JS's concat), attached in Additional Content. This node can be generalized to not just ControlNet type, but I just wanted to get going.
I'd love to have a way to coordinate indexes across graph invocations so I can grab the previous image and use that for context to make sequences, but hey, what can ya' do.
### Alternatives
_No response_
### Additional Content
```py
from typing import Union
from .baseinvocation import BaseInvocation, BaseInvocationOutput, Input, InputField, InvocationContext, invocation, UIType, OutputField, invocation_output
from .controlnet_image_processors import ControlField
class FieldDescriptions:
control = "An input ControlNet or List[ControlNet]"
@invocation_output("JoinControlNetsOutput")
class JoinControlNetsOutput(BaseInvocationOutput):
collection: list[ControlField] = OutputField(
description="The list of ControlNets for inference", title="ControlNets", ui_type=UIType.Collection
)
@invocation("JoinControlNetsInvocation", title="Join ControlNets", tags=["collection", "controlnet", "i2i", "img2img"], category="collections")
class JoinControlNetsInvocation(BaseInvocation):
"""Concatenate 2 input ControlNet or List[ControlNet] inputs"""
first: Union[ControlField, list[ControlField]] = InputField(
default=None, description=FieldDescriptions.control, input=Input.Connection, ui_order=0
)
second: Union[ControlField, list[ControlField]] = InputField(
default=None, description=FieldDescriptions.control, input=Input.Connection, ui_order=1
)
def invoke(self, context: InvocationContext) -> JoinControlNetsOutput:
control_list = []
self.concat_field(control_list, self.first)
self.concat_field(control_list, self.second)
return JoinControlNetsOutput(collection=control_list)
def concat_field(self, control_list: list[ControlField], input: Union[ControlField, list[ControlField]]):
if isinstance(input, ControlField):
control_list.append(input)
elif isinstance(input, list):
control_list.extend(input)
return control_list
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.