Comfy-Org / Comfy-Org/ComfyUI

[Feature Request:] Allow prompt batching

Open
#1,561 10 comments 10 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

I've been experimenting with batch generation and works fine with the Image Batch and Mask Batch of the `WAS_Node_Suite.py`. However, there is not a way to handle batches of text inputs. I've modified the encode method in ComfyUI/nodes.py, and it appears to be working as expected.

```py
class CLIPTextEncode:
@classmethod
def INPUT_TYPES(s):
return {"required": {"text": ("STRING", {"multiline": True}), "clip": ("CLIP", )}}
RETURN_TYPES = ("CONDITIONING",)
FUNCTION = "encode"

CATEGORY = "conditioning"

def encode(self, clip, text):
if isinstance(text, list):
batched_cond = []
batched_pooled = []

for single_text in text:
tokens = clip.tokenize(single_text)
cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True)
batched_cond.append(cond)
batched_pooled.append({"pooled_output": pooled})

# Stack along a new dimension to create a batched tensor
batched_cond_tensor = torch.stack(batched_cond, dim=0).squeeze(1)

return ([[batched_cond_tensor, {"pooled_output": batched_pooled}]], )
else:
tokens = clip.tokenize(text)
cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True)
return ([[cond, {"pooled_output": pooled}]], )
```
Would you be interested in a Pull Request for this change?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.