is there any plan to expand the func of fuse_lora_weights to peft package?
Open
@zerollzeng is already working on this.
Since Nov 24, 2023.
triaged
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Currently, the lora format supported by this function is too single. Can it support lora fusion implemented by the peft library? in demo/Diffusion
such as:
from peft import LoraConfig, get_peft_model, get_peft_model_state_dict
lora_config = LoraConfig(
r=args.lora_rank,
target_modules=[
"to_q",
"to_k",
"to_v",
"to_out.0",
"proj_in",
"proj_out",
"ff.net.0.proj",
"ff.net.2",
"conv1",
"conv2",
"conv_shortcut",
"downsamplers.0.conv",
"upsamplers.0.conv",
"time_emb_proj",
],
)
unet = get_peft_model(unet, lora_config)
# unet.add_adapter(adapter_name, lora_config )
# unet.load_adapter(unet_sub_dir, adapter_name=adapter_name)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.