huggingface / huggingface/diffusers

[New feature] A Noise Injection Method for Flux

Open
#10,071 10 comments 1 reaction 0 assignees View on GitHub
consider-for-modular-diffusers wip
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### New Feature for FluxPipeline
Paper: [Link](https://openreview.net/pdf?id=KzokzKV4JK)
Code : [Link](https://anonymous.4open.science/r/SSNI-F746/README.md)
workflow : [Link](https://www.dropbox.com/scl/fi/hhitjx6lqpqpv8xjx9ikq/Flux-Noise-Injection.json?rlkey=45xnu45j1i5owiwhc7z1hppn1&e=1&dl=0)
This paper introduces Sample-specific Score-aware Noise Injection (SSNI) to improve diffusion-based purification (DBP) methods. Unlike existing approaches that use a fixed noise level (t*) for all samples, SSNI adapts t* based on how noisy or clean each sample is. Using a pre-trained score network, SSNI estimates a sample's deviation from the clean data and adjusts the noise level accordingly.

This have stunning Results with Flux

![spider_with_noise_injection](https://github.com/user-attachments/assets/e87d3da6-6ff5-46c1-b0ff-20c00ad4e541)
![spider_sh](https://github.com/user-attachments/assets/e0f7a469-ae99-4095-9c9d-8cc88ed4691d)

@sayakpaul @yiyixuxu

Contributor guide

Open the contributing guide

Research direction

Start with FluxPipeline and read the linked paper, reference implementation, and workflow to understand the proposed SSNI method. Done would be an integrated noise-injection feature for Flux with validation against the behavior and results described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.