lllyasviel / lllyasviel/LayerDiffuse

Yet another unofficial Diffuser support

Open
#32 4 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
2.2k
Forks
36
PR merge metrics
No merged PRs in 30d

Description

Github: https://github.com/rootonchair/diffuser_layerdiffuse

This project is a port to Diffusers, it allows you to run transparent image with SD1.5 (transparent only or joint generation) and SDXL (Attn and Conv Injection) with Diffusers friendly API

Don't hesitate to give it a try

```python
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
import torch

from diffusers import StableDiffusionPipeline

from models import TransparentVAEDecoder
from loaders import load_lora_to_unet

model_path = hf_hub_download(
'LayerDiffusion/layerdiffusion-v1',
'layer_sd15_vae_transparent_decoder.safetensors',
)

vae_transparent_decoder = TransparentVAEDecoder.from_pretrained("digiplay/Juggernaut_final", subfolder="vae", torch_dtype=torch.float16).to("cuda")
vae_transparent_decoder.set_transparent_decoder(load_file(model_path))

pipeline = StableDiffusionPipeline.from_pretrained("digiplay/Juggernaut_final", vae=vae_transparent_decoder, torch_dtype=torch.float16, safety_checker=None).to("cuda")

model_path = hf_hub_download(
'LayerDiffusion/layerdiffusion-v1',
'layer_sd15_transparent_attn.safetensors'
)

load_lora_to_unet(pipeline.unet, model_path, frames=1)

image = pipeline(prompt="a dog sitting in room, high quality",
width=512, height=512,
num_images_per_prompt=1, return_dict=False)[0]
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the linked diffuser_layerdiffuse project and the shown Python entry points, including TransparentVAEDecoder and load_lora_to_unet. Clarify whether this repository should adopt the Diffusers-friendly API and which SD1.5 or SDXL support is in scope; done should be defined before implementation begins.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.