huggingface / huggingface/diffusers

Add ReNeg: An end-toend method designed to learn improved Negative embeddings (CVPR 2025 Highlight)

Open
#11,256 2 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### **Model/Pipeline/Scheduler description**

[ReNeg] is a reward-guided approach that directly learns Negative embeddings through gradient descent. The negative embedding learned within the same text embedding space exhibits strong generalization capabilities.

For example, using the same CLIP text encoder, the negative embedding learned on SD1.5 can be seamlessly transferred to text-to-image or even text-to-video models such as ControlNet, ZeroScope, and VideoCrafter2, resulting in consistent performance improvements across the board.

![Image](https://github.com/user-attachments/assets/9c5e653e-989c-48bf-bc31-1547513a1362)

### **Open Source Status**

- [x] The model implementation is available.
- [x] The model weights are available (Only relevant if addition is not a scheduler).

### **Provide useful links for the implementation**

Official implementation: https://github.com/AMD-AIG-AIMA/ReNeg

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.