lllyasviel / lllyasviel/stable-diffusion-webui-forge

Upgrading prompt inference models

Open
#1,601 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

The current reverse prompt inference model, based on natural language and Booru tags, has undergone numerous iterations. However, the integrated Clip and Deepdanbooru's performance and knowledge cut-off date now appear somewhat outdated. I've noticed that using JoyCaption to infer prompts from images, and then using the Flux model to regenerate images, can produce remarkably similar results to the original images. Perhaps should consider replacing the existing Deepdanbooru with the newer WD14 tagger model, and larger models that are difficult to integrate, such as JoyCaption, could be integrated into the Florence-2 space, creating a general reverse inference space?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the existing Clip and Deepdanbooru reverse prompt inference integrations. Compare the proposed WD14 tagger and the JoyCaption or Florence-2 approaches described in the issue, including their integration constraints. The issue does not define a specific entry point, acceptance criteria, or a settled implementation scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.