Comfy-Org / Comfy-Org/ComfyUI

Tensor Latents for multi image input.

Open
#8,896 1 comment 0 reactions 0 assignees View on GitHub
bug-cop:non-blocking Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Feature Idea

I used to think this is how latents work, why else a fancy name for an input image I thought, but to my surprise, not even OpenAI seems to do it this way, based on the type of errors I get from them. I think maybe Midjourney does, but regardless.

Something I lack is a translation of an input images a latent into a code block of model interpretable data. A prechewed data chuck that can recreate the image that was put in, but can be used to output it cohesively and adaptably to a new format, this so that you can use this sort of Tensor like data structure as a latent you can combine with other latents to get a cohesive image.

When using latents in Comfy I always get images just overlayed and half combined or not at all. It seems difficult to manage proper cross combinations of images to something cohesive. I think maybe Imagen 4 has this capability, but I am not sure.

I am rather asking if anyone knows a paper for such a thing so I can build it or if there is something like it already, or do I myself need to find a way to create TensorLatents myself?

### Existing Solutions

_No response_

### Other

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.