Comfy-Org / Comfy-Org/ComfyUI

How to add attention scale parameter to custom lora model?

Open
#6,836 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

Thanks for the great tool.

Recently, I trained a slider lora model based on FLUX, which can be loaded and used using the diffusers library, and can also be put into **Comfy/models/loras**.

**The question is how can I add a slider scale parameter to this node?**

I have searched for a long time in the official documentation, Issues, Discussions and Discord, but still can't find how to do this.

This parameter corresponds to the line `joint_attention_kwargs={"scale": scale * 1 / 16},` in the code below:

```python
from pathlib import Path

import torch
from diffusers import FluxPipeline

lora_path = "outputs/flux-hair_sliders/weights/flux-hair_sliders_latest_multiplied_1.0.safetensors"
pipe = FluxPipeline.from_pretrained("models/FLUX.1-dev", torch_dtype=torch.bfloat16)
pipe.to("cuda:1")
pipe.load_lora_weights(lora_path)

scales = (-5, -2.5, 0, 2.5, 5)
prompt = "A fisherman wearing a worn cap and a thick sweater, net slung over his shoulder, face weathered by the sea; a lively harbor at dawn."
for scale in scales:
out = pipe(
prompt=prompt,
guidance_scale=3.5,
height=512,
width=512,
num_inference_steps=25,
joint_attention_kwargs={"scale": scale * 1 / 16},
generator=torch.Generator().manual_seed(42),
).images[0]

save_img_path = f"{time_stamp}_scale_{scale}.jpg"
out.save(save_img_path)
```

Any response is appreciated.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.