deepseek-ai / deepseek-ai/Janus
Does Janus' image generation mode support multi-modal input?
- Dominant language
- Python
- Stars
- 17.8k
- Forks
- 2.2k
- PR merge metrics
- No merged PRs in 30d
Description
Hi @learningpro @gnobitab @TheOneTrueGuy @mowentian @soloice , thanks for your excellent work!
I recently tried to use Janus for **image generation**.
In addition to providing text prompts, I also provided **other images as part of the prompt**. However, in the official implementation of Janus by transformers, I found the following code in the definition of **JanusProcessor**, which seems to mean that when generating images, the processor does not process the image elements in the input? So it seems that in image generation mode, the model only processes unimodal text prompts?
> # Process images if pixel values are provided.
> if images is not None and generation_mode != "image":
> data["pixel_values"] = self.image_processor(images=images, **output_kwargs["images_kwargs"])[
> "pixel_values"
> ]
Therefore, I hope to confirm with the official whether the image generation part of Janus supports **multimodal input**?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.