huggingface / huggingface/diffusers

Support multiple control nets in the `StableDiffusionControlNetXSPipeline`/`StableDiffusionXLControlNetXSPipeline`

Abierto
#8,434 12 comentarios 0 reacciones 0 asignados Ver en GitHub
community-examples contributions-welcome good first issue
Lenguaje dominante
Python
Estrellas
34.5k
Forks
7.3k
Merge medio
3 d 3 h
PR fusionados (30 d)
91

Descripción

**Is your feature request related to a problem? Please describe.**
The `StableDiffusionControlNetPipeline`/`StableDiffusionXLControlNetPipeline` allow you to specify a list of control nets (i.e. depth and canny)

However, the new `ControlNetXS` pipelines only allow you provide a single control net

**Describe the solution you'd like.**
I want to be able to do:
```python
vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
canny_controlnet = ControlNetXSAdapter.from_pretrained(
"UmerHA/Testing-ConrolNetXS-SDXL-canny", torch_dtype=torch.float16
)
depth_controlnet = ControlNetXSAdapter.from_pretrained(
"UmerHA/Testing-ConrolNetXS-SDXL-depth", torch_dtype=torch.float16
)
pipe = StableDiffusionXLControlNetXSPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0", controlnet=[canny_controlnet, depth_controlnet], torch_dtype=torch.float16
)
```

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Start with the StableDiffusionControlNetXSPipeline and StableDiffusionXLControlNetXSPipeline entry points, then compare their control-net handling with StableDiffusionControlNetPipeline and StableDiffusionXLControlNetPipeline. Done means both ControlNetXS pipelines accept a list such as canny and depth and support the usage shown in the issue.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
python, pytorch
Área
ai, machine-learning
Tipo de issue
Nueva funcionalidad
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Activo
Claridad
Bastante claro
Aptitud para principiantes
55/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.