would it be possible to try to add DeepSeek Janus Pro's text_encode configuration
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Feature Idea
Since DeepSeek Janus Pro has very good image comprehension, would it be possible to try to add Janus Pro's text_encode configuration and use it to comprehend the image and adjust the ClipTextEncode appropriately before handing it over to the Flux model for image modification, perhaps getting a strong Edit effect boost?
### Existing Solutions
Currently analyzed Janus Pro's model, using llama's tokenizer, but the number of layers and the shape of each layer is a bit confusing as to how ComfyUI converts it.
I made my own object parser to parse Janus Pro's model objects (the two core models, their respective layers and the torch module they call) see below:
https://github.com/comfyanonymous/ComfyUI/issues/6640#issuecomment-2635637873
### Other
I put it directly into the CLIP Extra Models directory, and was able to execute it smoothly, but I can find that the text does not present a successful CLIP. so I think that I should need to match up the different layers, but I haven't messed with this problem at all, and it's really a bit of a headache.
Contributor guide
Assessment
This issue has not been assessed yet.