stable_cascade model/pipeline review

stable_cascade model/pipeline review

Commit tested: 0f1abc4ae8b0eb2a3b40e82a310507281144c423

Review performed against the repository review rules.

Reviewed: public imports/lazy loading, config/loading/single-file surface, runtime dtype/device paths, offload/xFormers delegation, model forward behavior, docs, fast/slow tests, and duplicate issues/PRs.

Duplicate search status: searched huggingface/diffusers Issues/PRs for stable_cascade, class names, prompt_embeds_pooled, image-conditioning inputs, latents dtype, timesteps, xFormers, and coverage. No exact duplicates found. Related but not duplicate: #7355/#7644 for older arbitrary-resolution dtype errors, and #7598 for bf16 image tensor preprocessing.

Issue 1: Single image conditioning inputs are advertised but rejected

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py#L262-L274
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py#L345-L351
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_combined.py#L162-L164

Problem:
StableCascadePriorPipeline and StableCascadeCombinedPipeline type-hint images as a single tensor/PIL image or a list, but check_inputs() uses if images: and then iterates images. A tensor raises ambiguous-bool errors, and a single PIL image raises 'Image' object is not iterable.

Impact:
Image-conditioned Stable Cascade calls fail for documented input forms unless users wrap the image in a list.

Reproduction:

from PIL import Image
import torch
from diffusers import DDPMWuerstchenScheduler, StableCascadePriorPipeline, StableCascadeUNet

prior = StableCascadeUNet(
    conditioning_dim=8, block_out_channels=(8,), num_attention_heads=(-1,),
    down_num_layers_per_block=(1,), up_num_layers_per_block=(1,),
    down_blocks_repeat_mappers=(1,), up_blocks_repeat_mappers=(1,),
    block_types_per_layer=(("SDCascadeResBlock", "SDCascadeTimestepBlock"),),
    clip_text_pooled_in_channels=8, clip_image_in_channels=8,
)
pipe = StableCascadePriorPipeline(None, None, prior, DDPMWuerstchenScheduler())

for images in [Image.new("RGB", (8, 8)), torch.zeros(3, 8, 8)]:
    try:
        pipe.check_inputs(prompt="cat", images=images)
    except Exception as e:
        print(type(e).__name__, str(e).splitlines()[0])

Relevant precedent:
Other image pipelines normalize single images to lists before iterating, for example pipeline_if_img2img.py.

Suggested fix:

if images is not None and not isinstance(images, list):
    images = [images]

Apply before validation and before encode_image().

Issue 2: Decoder precomputed prompt embeds break CFG unless negative pooled embeds are also supplied

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py#L241-L280
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py#L422-L440

Problem:
The decoder only calls encode_prompt() when both prompt_embeds and negative_prompt_embeds are None. If a user supplies prompt_embeds/prompt_embeds_pooled and sets guidance_scale > 1, the pipeline never creates negative_prompt_embeds_pooled, then torch.cat() receives None.

Impact:
The documented precomputed-embedding path fails for classifier-free guidance instead of generating empty negative embeddings or raising a clear validation error.

Reproduction:

import torch
from diffusers import DDPMWuerstchenScheduler, StableCascadeDecoderPipeline, StableCascadeUNet

decoder = StableCascadeUNet(
    in_channels=4, out_channels=4, conditioning_dim=8, block_out_channels=(8,),
    num_attention_heads=(-1,), down_num_layers_per_block=(1,), up_num_layers_per_block=(1,),
    down_blocks_repeat_mappers=(1,), up_blocks_repeat_mappers=(1,),
    block_types_per_layer=(("SDCascadeResBlock", "SDCascadeTimestepBlock"),),
    clip_text_pooled_in_channels=8, effnet_in_channels=4,
)
pipe = StableCascadeDecoderPipeline(decoder, None, None, DDPMWuerstchenScheduler(), None, latent_dim_scale=1.0)

try:
    pipe(
        image_embeddings=torch.randn(1, 4, 1, 1),
        prompt_embeds=torch.randn(1, 77, 8),
        prompt_embeds_pooled=torch.randn(1, 1, 8),
        guidance_scale=2.0,
        num_inference_steps=1,
        output_type="latent",
    )
except Exception as e:
    print(type(e).__name__, str(e).splitlines()[0])

Relevant precedent:
StableCascadePriorPipeline.check_inputs() already validates pooled prompt embeddings.

Suggested fix:

if prompt_embeds is not None and prompt_embeds_pooled is None:
    raise ValueError("If `prompt_embeds` are provided, `prompt_embeds_pooled` must also be provided.")
if negative_prompt_embeds is not None and negative_prompt_embeds_pooled is None:
    raise ValueError("If `negative_prompt_embeds` are provided, `negative_prompt_embeds_pooled` must also be provided.")

if self.do_classifier_free_guidance and negative_prompt_embeds_pooled is None:
    _, _, _, negative_prompt_embeds_pooled = self.encode_prompt(...)

Issue 3: StableCascadeUNet.get_clip_embeddings() mishandles 2D pooled text embeddings

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/unets/unet_stable_cascade.py#L432-L448

Problem:
The method detects 2D clip_txt_pooled and assigns clip_txt_pool = clip_txt_pooled.unsqueeze(1), but then ignores that normalized tensor and maps/views the original 2D tensor.

Impact:
Users passing normal CLIP pooled embeddings shaped (batch, dim) hit a runtime shape error, even though the method appears intended to support that shape.

Reproduction:

import torch
from diffusers import StableCascadeUNet

model = StableCascadeUNet(
    conditioning_dim=8, block_out_channels=(8,), num_attention_heads=(-1,),
    down_num_layers_per_block=(1,), up_num_layers_per_block=(1,),
    down_blocks_repeat_mappers=(1,), up_blocks_repeat_mappers=(1,),
    block_types_per_layer=(("SDCascadeResBlock", "SDCascadeTimestepBlock"),),
    clip_text_pooled_in_channels=8,
)

try:
    model.get_clip_embeddings(torch.randn(1, 8))
except Exception as e:
    print(type(e).__name__, str(e).splitlines()[0])

Relevant precedent:
The pipelines produce pooled embeddings as (batch, 1, dim) internally; public direct model use should normalize 2D inputs to that same shape.

Suggested fix:

if clip_txt_pooled.ndim == 2:
    clip_txt_pooled = clip_txt_pooled.unsqueeze(1)

clip_txt_pool = self.clip_txt_pooled_mapper(clip_txt_pooled).view(
    clip_txt_pooled.size(0), clip_txt_pooled.size(1) * self.config.clip_seq, -1
)

Issue 4: sca/crp conditioning tensors crash batch forwards

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/unets/unet_stable_cascade.py#L555-L564

Problem:
forward() uses t_cond = cond or torch.zeros_like(timestep_ratio). Tensor truthiness is invalid for batched tensors.

Impact:
The model exposes sca and crp conditioning parameters but cannot use them for normal batched inputs.

Reproduction:

import torch
from diffusers import StableCascadeUNet

model = StableCascadeUNet(
    in_channels=4, out_channels=4, conditioning_dim=8, block_out_channels=(8,),
    num_attention_heads=(-1,), down_num_layers_per_block=(1,), up_num_layers_per_block=(1,),
    down_blocks_repeat_mappers=(1,), up_blocks_repeat_mappers=(1,),
    block_types_per_layer=(("SDCascadeResBlock", "SDCascadeTimestepBlock"),),
    clip_text_pooled_in_channels=8,
)

try:
    model(
        sample=torch.randn(2, 4, 8, 8),
        timestep_ratio=torch.ones(2),
        clip_text_pooled=torch.randn(2, 1, 8),
        sca=torch.tensor([0.1, 0.2]),
        crp=torch.tensor([0.3, 0.4]),
    )
except Exception as e:
    print(type(e).__name__, str(e).splitlines()[0])

Relevant precedent:
Standard tensor optional handling uses explicit is None checks.

Suggested fix:

t_cond = cond if cond is not None else torch.zeros_like(timestep_ratio)

Issue 5: User-supplied latents are moved to device but not cast to pipeline dtype

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py#L145-L152
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py#L124-L131

Problem:
Both prepare_latents() methods cast generated latents to dtype, but pre-generated latents only call .to(device). Passing float32 latents to bf16/fp16 weights preserves float32 and can fail in convolutions.

Impact:
Reusing deterministic latents with half/bfloat16 Stable Cascade can error or silently change compute behavior. Related dtype-error issues exist (#7355/#7644), but those concerned arbitrary-resolution interpolation, not this supplied-latents path.

Reproduction:

import torch
from diffusers import DDPMWuerstchenScheduler, StableCascadePriorPipeline, StableCascadeUNet

prior = StableCascadeUNet(
    conditioning_dim=8, block_out_channels=(8,), num_attention_heads=(-1,),
    down_num_layers_per_block=(1,), up_num_layers_per_block=(1,),
    down_blocks_repeat_mappers=(1,), up_blocks_repeat_mappers=(1,),
    block_types_per_layer=(("SDCascadeResBlock", "SDCascadeTimestepBlock"),),
    clip_text_pooled_in_channels=8, clip_image_in_channels=8,
)
pipe = StableCascadePriorPipeline(None, None, prior, DDPMWuerstchenScheduler())
latents = torch.randn(1, prior.config.in_channels, 1, 1, dtype=torch.float32)

out = pipe.prepare_latents(1, 42, 42, 1, torch.bfloat16, torch.device("cpu"), None, latents, pipe.scheduler)
print(out.dtype)  # torch.float32, expected torch.bfloat16

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/flux/pipeline_flux.py#L615-L617

Suggested fix:

latents = latents.to(device=device, dtype=dtype)

Issue 6: StableCascadePriorPipeline.timesteps is public but ignored

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py#L384
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py#L558-L560
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_combined.py#L222-L229

Problem:
The prior pipeline accepts timesteps, and the combined docstring advertises customized prior_timesteps/timesteps, but the prior always calls self.scheduler.set_timesteps(num_inference_steps, device=device).

Impact:
Users cannot control the prior schedule despite the public API suggesting they can.

Reproduction:

import inspect
from diffusers import StableCascadePriorPipeline

src = inspect.getsource(StableCascadePriorPipeline.__call__)
print("timesteps parameter exists:", "timesteps: list[float]" in src)
print("passed to scheduler:", "timesteps=timesteps" in src)

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/schedulers/scheduling_ddpm_wuerstchen.py#L141-L160

Suggested fix:

self.scheduler.set_timesteps(num_inference_steps, timesteps=timesteps, device=device)

Also either add matching decoder/combined args or remove the combined docstring claims.

Issue 7: Combined xFormers enable only reaches the decoder

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_combined.py#L127-L128

Problem:
StableCascadeCombinedPipeline.enable_xformers_memory_efficient_attention() delegates only to decoder_pipe, leaving the prior unchanged.

Impact:
Users enabling xFormers on the combined pipeline do not get memory-efficient attention for the prior stage.

Reproduction:

from unittest.mock import Mock
from diffusers import StableCascadeCombinedPipeline

pipe = StableCascadeCombinedPipeline.__new__(StableCascadeCombinedPipeline)
pipe.prior_pipe = Mock()
pipe.decoder_pipe = Mock()

pipe.enable_xformers_memory_efficient_attention()
print(pipe.prior_pipe.enable_xformers_memory_efficient_attention.call_count)   # 0
print(pipe.decoder_pipe.enable_xformers_memory_efficient_attention.call_count) # 1

Relevant precedent:
The prior and decoder both contain StableCascadeUNet attention modules and both support recursive xFormers enabling through ModelMixin.

Suggested fix:

def enable_xformers_memory_efficient_attention(self, attention_op: Callable | None = None):
    self.prior_pipe.enable_xformers_memory_efficient_attention(attention_op)
    self.decoder_pipe.enable_xformers_memory_efficient_attention(attention_op)

Issue 8: Test coverage misses combined slow coverage and dedicated model fast tests

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/pipelines/stable_cascade/test_stable_cascade_combined.py#L33-L50
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/pipelines/stable_cascade/test_stable_cascade_combined.py#L242-L244
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/single_file/test_model_sd_cascade_unet_single_file.py#L31-L39

Problem:
Prior and decoder have slow integration tests, but the combined pipeline has no @slow test. StableCascadeUNet also lacks a dedicated fast tests/models/... test; it is mostly covered indirectly and by slow single-file config checks.

Impact:
The issues above are not covered by the current fast/slow matrix, especially direct model input-shape behavior and combined pipeline delegation behavior.

Reproduction:

from pathlib import Path

print("model fast tests:", list(Path("tests/models").glob("**/*stable*cascade*.py")))
for path in sorted(Path("tests/pipelines/stable_cascade").glob("test_*.py")):
    print(path.name, "@slow" in path.read_text())

Relevant precedent:
Most active model families have dedicated ModelTesterMixin coverage for forward, save/load, dtype, gradient checkpointing, and attention/offload behavior.

Suggested fix:
Add:

# tests/models/unets/test_models_unet_stable_cascade.py
class StableCascadeUNetTests(ModelTesterMixin, unittest.TestCase):
    model_class = StableCascadeUNet

and a @slow combined pipeline integration test that loads stabilityai/stable-cascade and exercises the full prior+decoder path.

Issue 9: Pipeline docstring examples are not runnable

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py#L43-L55
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py#L38-L54

Problem:
The prior example assigns prior_pipe but calls pipe(prompt). The decoder example calls StableCascadeDecoderPipeline.from_pretrain instead of from_pretrained and also calls pipe(prompt).

Impact:
Autodoc examples copied by users fail immediately.

Reproduction:

from pathlib import Path

for path in [
    Path("src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py"),
    Path("src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py"),
]:
    text = path.read_text()
    print(path.name, "from_pretrain(" in text, "prior_output = pipe(prompt)" in text)

Relevant precedent:
The docs page docs/source/en/api/pipelines/stable_cascade.md uses the correct top-level imports and from_pretrained.

Suggested fix:

prior_output = prior_pipe(prompt)
gen_pipe = StableCascadeDecoderPipeline.from_pretrained(...)

Start with the supplied reproductions and inspect the affected entry points in pipeline_stable_cascade_prior.py, pipeline_stable_cascade_combined.py, pipeline_stable_cascade.py, and unet_stable_cascade.py. Compare their input normalization, embedding, conditioning, latent, and timestep handling with the cited precedents, then add regression coverage showing that all six documented cases work as advertised.