CarperAI / CarperAI/DRLX

Add support for BitFit

Open
#31 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
193
Forks
8
PR merge metrics
No merged PRs in 30d

Description

Paper: https://aclanthology.org/2022.acl-short.1/

Summary (my words):

As a model trainer, it would be nice if we could use this directed policy optimization trainer to train just the bias of the U-net, keeping the weights frozen.

Initial testing shows that this approach allows us to carefully direct the model toward better details / aesthetics while maintaining most of the model's core structure.

Where full weight **and** bias tuning results in almost complete destruction of SD 2.1-v using just 8 images for finetuning, this method allows pushing past 400 epochs on the same dataset.

Example:

![image](https://github.com/CarperAI/DRLX/assets/59658056/d155a5aa-81e9-4386-b508-c6704cce3b64)
The starting point ^

![image](https://github.com/CarperAI/DRLX/assets/59658056/4b13fddd-36de-4852-899c-e6870a88831e)
After just 810 steps ^

This is without any DPO, simply finetuning based on MSE loss and velocity objective.

Comparison, the mode collapse of SD 2.1-v when tuning weights **and** bias which occurs in fewer steps:

![image](https://github.com/CarperAI/DRLX/assets/59658056/adf6f86b-7dcb-4295-8fce-37df224e6ace)

This is using the same hyperparameters, eg. learning rate/scheduler/dataset/seeds.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.