invoke-ai / invoke-ai/InvokeAI

[enhancement]: Concat ControlNet(s) for easier multiple control nets into a denoise latent

Open
#4,526 5 comments 0 reactions 1 assignee Claimed by @psychedelicious View on GitHub
controlnet enhancement generation nodes / workflows
Dominant language
Python
Stars
28.2k
Forks
3k
Avg merge
6d 5h
Merged PRs (30d)
19

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Contact Details

_No response_

### What should this feature add?

In workflows, the `DenoiseLatents` steps takes a ControlNet or a List[ControlNet], but the only way to get one (that I found) is via a collect -- this doesn't work when you're inside of an iteration, as it collects EVERY control net before passing them along. You can have an iterated control net off of, say, [a video](https://github.com/helix4u/load_video_frame), and then have a fixed set of control nets to keep the context like a background or color palette.

Here's a more or less generic `Concat` node that does `concat(T|T[], T|T[]): T[]` (similar to JS's concat), attached in Additional Content. This node can be generalized to not just ControlNet type, but I just wanted to get going.

I'd love to have a way to coordinate indexes across graph invocations so I can grab the previous image and use that for context to make sequences, but hey, what can ya' do.

### Alternatives

_No response_

### Additional Content

```py
from typing import Union

from .baseinvocation import BaseInvocation, BaseInvocationOutput, Input, InputField, InvocationContext, invocation, UIType, OutputField, invocation_output
from .controlnet_image_processors import ControlField

class FieldDescriptions:
control = "An input ControlNet or List[ControlNet]"

@invocation_output("JoinControlNetsOutput")
class JoinControlNetsOutput(BaseInvocationOutput):
collection: list[ControlField] = OutputField(
description="The list of ControlNets for inference", title="ControlNets", ui_type=UIType.Collection
)

@invocation("JoinControlNetsInvocation", title="Join ControlNets", tags=["collection", "controlnet", "i2i", "img2img"], category="collections")
class JoinControlNetsInvocation(BaseInvocation):
"""Concatenate 2 input ControlNet or List[ControlNet] inputs"""

first: Union[ControlField, list[ControlField]] = InputField(
default=None, description=FieldDescriptions.control, input=Input.Connection, ui_order=0
)
second: Union[ControlField, list[ControlField]] = InputField(
default=None, description=FieldDescriptions.control, input=Input.Connection, ui_order=1
)

def invoke(self, context: InvocationContext) -> JoinControlNetsOutput:
control_list = []
self.concat_field(control_list, self.first)
self.concat_field(control_list, self.second)
return JoinControlNetsOutput(collection=control_list)

def concat_field(self, control_list: list[ControlField], input: Union[ControlField, list[ControlField]]):
if isinstance(input, ControlField):
control_list.append(input)
elif isinstance(input, list):
control_list.extend(input)
return control_list
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.