Comfy-Org / Comfy-Org/ComfyUI

Neural Network Compression Framework (NNCF) -> VRAM saving

Open
#6,738 0 comments 0 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

### Feature Idea

It would be nice to have like in sdnext as it says here [https://vladmandic.github.io/sdnext-docs/NNCF-Compression/](url), it saves lots of VRAM.

From sdnext wiki.

Features

Uses INT8, halves the model size
Saves 3.4 GB of VRAM with SDXL

Disadvantages

It is Autocast, GPU will still use 16 Bit to run the model and will be slower
Not implemented in Original backend
Fused projections are not compatible with NNCF
Using Loras will make generations slower
Options

These results compares NNCF 8 bit to 16 bit.

Model:
Compresses UNet or Transformers part of the model.
This is where the most memory savings happens for Stable Diffusion.

SDXL: 2500 MB~ memory savings.
SD 1.5: 750 MB~ memory savings.
PixArt-XL-2: 600 MB~ memory savings.

Text Encoder:
Compresses Text Encoder parts of the model.
This is where the most memory savings happens for PixArt.

PixArt-XL-2: 4750 MB~ memory savings.
SDXL: 750 MB~ memory savings.
SD 1.5: 120 MB~ memory savings.

VAE:
Compresses VAE part of the model.
Memory savings from compressing VAE is pretty small.

SD 1.5 / SDXL / PixArt-XL-2: 75 MB~ memory savings.

### Existing Solutions

_No response_

### Other

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.