Tensor Latents for multi image input.
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Feature Idea
I used to think this is how latents work, why else a fancy name for an input image I thought, but to my surprise, not even OpenAI seems to do it this way, based on the type of errors I get from them. I think maybe Midjourney does, but regardless.
Something I lack is a translation of an input images a latent into a code block of model interpretable data. A prechewed data chuck that can recreate the image that was put in, but can be used to output it cohesively and adaptably to a new format, this so that you can use this sort of Tensor like data structure as a latent you can combine with other latents to get a cohesive image.
When using latents in Comfy I always get images just overlayed and half combined or not at all. It seems difficult to manage proper cross combinations of images to something cohesive. I think maybe Imagen 4 has this capability, but I am not sure.
I am rather asking if anyone knows a paper for such a thing so I can build it or if there is something like it already, or do I myself need to find a way to create TensorLatents myself?
### Existing Solutions
_No response_
### Other
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.