lllyasviel / lllyasviel/stable-diffusion-webui-forge

GGUF Q8_0 and Automatic Diffusion (8bit LoRa) generating poor quality

Open
#1,624 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Since ba01ad37, LoRas loaded in 8bit to the Q8_0 GGUF generate to a poor quality. Loading the LoRa in 16bit appears to fix this issue, but there are subtle differences in the generations from rounding.

This does not seem to happen with FP8 safetensor or with the NF4 - just the Q8_0 GGUF.
This does not happen with checkpoint ba01ad37 and earlier.

Example at commit 3b9b2f65:
` A pixelart drawing of a chicken`

Diffusion in Low bits set to Automatic:
![tmpdhprdqqr](https://github.com/user-attachments/assets/6d05ecfe-5d0e-43f4-99ea-0c6729ddb94a)

Diffusion in Low bits set to Automatic (FP16 Lora)
![tmpez8qh4ed](https://github.com/user-attachments/assets/6584e810-1c2b-487a-9cfa-aba46cfcedd4)

Example at commit ba01ad37 (last testable commit before 8bit LoRa changed):

Diffusion in Low bits set to Automatic:
![tmpw1eub1xa](https://github.com/user-attachments/assets/fa5c2467-f038-45be-bc9e-213fa50e5835)

Diffusion in Low bits set to Automatic (FP16 Lora):
![tmpfoodnrpo](https://github.com/user-attachments/assets/e1922898-5545-4fe2-855b-6ec6429f5220)

Generations with the FP8 safetensor for comparison:

Diffusion in Low bits set to Automatic:
![tmpyjgjvqxv](https://github.com/user-attachments/assets/1b815346-6c2e-43f5-970b-c98447d89cd8)

Diffusion in Low bits set to Automatic (FP16 Lora):
![tmpmhzlqzzk](https://github.com/user-attachments/assets/f64a5260-829f-4f1c-9564-7a530088950a)

And here is the NF4 for comparison:

Diffusion in Low bits set to Automatic:
![tmpmaz2i3rl](https://github.com/user-attachments/assets/9ad0864e-173d-4a5e-890a-a2c0cdb97d71)

Diffusion in Low bits set to Automatic (FP16 Lora):
![tmph9qotj7_](https://github.com/user-attachments/assets/5e5c6c39-9e4a-466f-8948-686d1671ea9c)

(LoRa used here was https://civitai.com/models/704779/flux-gen-5-trainer-sprites but appears to happen with all LoRas)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the example with Q8_0 GGUF, Automatic diffusion low bits, and an 8bit LoRa at commits 3b9b2f65 and ba01ad37, then compare the generated results with FP16 LoRa, FP8 safetensor, and NF4. Done means the Q8_0 GGUF 8bit LoRa generation no longer has the reported quality degradation without requiring the FP16 workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.