lllyasviel / lllyasviel/stable-diffusion-webui-forge
Upgrading prompt inference models
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
The current reverse prompt inference model, based on natural language and Booru tags, has undergone numerous iterations. However, the integrated Clip and Deepdanbooru's performance and knowledge cut-off date now appear somewhat outdated. I've noticed that using JoyCaption to infer prompts from images, and then using the Flux model to regenerate images, can produce remarkably similar results to the original images. Perhaps should consider replacing the existing Deepdanbooru with the newer WD14 tagger model, and larger models that are difficult to integrate, such as JoyCaption, could be integrated into the Florence-2 space, creating a general reverse inference space?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the existing Clip and Deepdanbooru reverse prompt inference integrations. Compare the proposed WD14 tagger and the JoyCaption or Florence-2 approaches described in the issue, including their integration constraints. The issue does not define a specific entry point, acceptance criteria, or a settled implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100