lumina model/pipeline review

# `lumina` model/pipeline review

Commit tested: `0f1abc4ae8b0eb2a3b40e82a310507281144c423`

Review performed against the repository review rules.

Reviewed target files, top-level/lazy exports, dummy exports, docs, fast/slow tests, dtype/device handling, offload-related config, and attention processor behavior. Fast model and pipeline tests exist, and a slow Lumina pipeline test exists. Targeted pytest collection was attempted with `.venv`, but the local Torch build is missing `torch._C._distributed_c10d`, so shared test mixins fail during collection before Lumina tests run.

Duplicate search status: focused `gh` and GitHub connector searches found no exact duplicates for the Lumina findings. Related but not exact duplicates: https://github.com/huggingface/diffusers/pull/10827, https://github.com/huggingface/diffusers/issues/13613, https://github.com/huggingface/diffusers/pull/11368.

## Issue 1: Deprecated alias is exported but cannot be constructed

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L940-L950
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/__init__.py#L624-L625

Problem:
`LuminaText2ImgPipeline` is still publicly exported, but its constructor calls `deprecate(..., "0.34", ...)`. Current version is `0.38.0.dev0`, so construction raises `ValueError` instead of warning.

Impact:
Users can import the backwards-compatible alias, but any path that instantiates it fails immediately.

Reproduction:
```python
from diffusers import LuminaText2ImgPipeline

LuminaText2ImgPipeline(None, None, None, None, None)
```

Relevant precedent:
Related rename PR, but it does not remove/fix the current exported alias failure: https://github.com/huggingface/diffusers/pull/10827

Suggested fix:
```python
# Either remove the alias from pipeline_lumina.py, lazy exports, top-level exports,
# and dummy objects, or bump the deprecation target to a future version if keeping it.
deprecate(
"diffusers.pipelines.lumina.pipeline_lumina.LuminaText2ImgPipeline",
"1.0.0",
deprecation_message,
)
```

## Issue 2: VAE scale factor is hardcoded

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L196-L197

Problem:
`vae_scale_factor` is hardcoded to `8` instead of derived from `vae.config.block_out_channels`.

Impact:
Custom/tiny VAEs compute wrong latent sizes, default image sizes, and input divisibility checks.

Reproduction:
```python
from diffusers import AutoencoderKL, FlowMatchEulerDiscreteScheduler, LuminaNextDiT2DModel, LuminaPipeline

transformer = LuminaNextDiT2DModel(sample_size=4, hidden_size=24, num_layers=1, num_attention_heads=3, num_kv_heads=1, multiple_of=16, learn_sigma=False, cross_attention_dim=32)
vae = AutoencoderKL(block_out_channels=(32, 64), down_block_types=("DownEncoderBlock2D", "DownEncoderBlock2D"), up_block_types=("UpDecoderBlock2D", "UpDecoderBlock2D"), latent_channels=4)
pipe = LuminaPipeline(transformer, FlowMatchEulerDiscreteScheduler(), vae, None, None)

print(pipe.vae_scale_factor) # 8
print(2 ** (len(vae.config.block_out_channels) - 1)) # 2
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/flux/pipeline_flux.py#L198-L212

Suggested fix:
```python
self.vae_scale_factor = 2 ** (len(self.vae.config.block_out_channels) - 1) if getattr(self, "vae", None) else 8
self.image_processor = VaeImageProcessor(vae_scale_factor=self.vae_scale_factor)
```

## Issue 3: `max_sequence_length` is ignored

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L206-L222
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L782-L794

Problem:
`__call__` accepts `max_sequence_length`, but `encode_prompt` only captures it in `**kwargs`, and `_get_gemma_prompt_embeds` always uses `self.max_sequence_length`.

Impact:
Users cannot shorten or adjust prompt tokenization through the documented pipeline argument.

Reproduction:
```python
import torch
from types import SimpleNamespace
from diffusers import LuminaPipeline

class Tok:
def __call__(self, prompt, **kw):
n = kw.get("max_length") or 11
return SimpleNamespace(input_ids=torch.zeros(len(prompt), n, dtype=torch.long), attention_mask=torch.ones(len(prompt), n))
def batch_decode(self, ids): return [""]

class Enc(torch.nn.Module):
dtype = torch.float32
def forward(self, input_ids, **kw):
h = torch.zeros(input_ids.shape[0], input_ids.shape[1], 8)
return SimpleNamespace(hidden_states=[h, h, h])

pipe = LuminaPipeline.__new__(LuminaPipeline)
pipe.max_sequence_length, pipe.tokenizer, pipe.text_encoder, pipe.transformer = 256, Tok(), Enc(), None
embeds, mask, *_ = pipe.encode_prompt("x", do_classifier_free_guidance=False, device=torch.device("cpu"), max_sequence_length=7)
print(embeds.shape, mask.shape) # torch.Size([1, 256, 8]) torch.Size([1, 256])
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina2/pipeline_lumina2.py#L192-L215

Suggested fix:
```python
def _get_gemma_prompt_embeds(..., max_sequence_length: int = 256):
...
max_length=max_sequence_length
...
removed_text = self.tokenizer.batch_decode(untruncated_ids[:, max_sequence_length - 1 : -1])

# and forward max_sequence_length from encode_prompt into _get_gemma_prompt_embeds
```

## Issue 4: Prompt conditioning expansion is wrong for multiple images

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L252-L257
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L310-L369

Problem:
Generated prompt masks use `repeat(num_images_per_prompt, 1)`, which orders masks as `[p1, p2, p1, p2]` while embeddings are `[p1, p1, p2, p2]`. Also, when users pass precomputed prompt/negative embeddings, `encode_prompt` does not expand them for `num_images_per_prompt`.

Impact:
Batched prompts with different mask lengths can pair embeddings with the wrong masks, and precomputed-embedding workflows fail or mis-broadcast when generating multiple images per prompt.

Reproduction:
```python
import torch
from diffusers import LuminaPipeline

pipe = LuminaPipeline.__new__(LuminaPipeline)
pe = torch.zeros(1, 5, 8)
pm = torch.ones(1, 5, dtype=torch.long)
ne = torch.ones(1, 5, 8)
nm = torch.ones(1, 5, dtype=torch.long)

out = pipe.encode_prompt(
prompt=None, do_classifier_free_guidance=True, num_images_per_prompt=2,
device=torch.device("cpu"), prompt_embeds=pe, prompt_attention_mask=pm,
negative_prompt_embeds=ne, negative_prompt_attention_mask=nm,
)
print(out[0].shape, out[2].shape) # both still batch 1; expected batch 2 each
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py#L390-L431

Suggested fix:
```python
bs, seq_len, _ = prompt_embeds.shape
prompt_embeds = prompt_embeds.repeat(1, num_images_per_prompt, 1).view(bs * num_images_per_prompt, seq_len, -1)
prompt_attention_mask = prompt_attention_mask.repeat(1, num_images_per_prompt).view(bs * num_images_per_prompt, -1)

if do_classifier_free_guidance:
bs, seq_len, _ = negative_prompt_embeds.shape
negative_prompt_embeds = negative_prompt_embeds.repeat(1, num_images_per_prompt, 1).view(bs * num_images_per_prompt, seq_len, -1)
negative_prompt_attention_mask = negative_prompt_attention_mask.repeat(1, num_images_per_prompt).view(bs * num_images_per_prompt, -1)
```

## Issue 5: Provided latents are not cast to the requested dtype

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L597-L615

Problem:
`prepare_latents` moves user-provided latents to the device but leaves dtype unchanged.

Impact:
Supplying float32 latents to a bf16/fp16 pipeline can feed mismatched activations into lower-precision transformer weights.

Reproduction:
```python
import torch
from diffusers import LuminaPipeline

pipe = LuminaPipeline.__new__(LuminaPipeline)
pipe.vae_scale_factor = 8
latents = torch.ones(1, 4, 2, 2, dtype=torch.float32)
out = pipe.prepare_latents(1, 4, 16, 16, torch.bfloat16, torch.device("cpu"), None, latents)
print(out.dtype) # torch.float32
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3.py#L633-L646

Suggested fix:
```python
else:
latents = latents.to(device=device, dtype=dtype)
```

## Issue 6: Lumina attention bypasses the attention backend dispatcher

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/lumina_nextdit2d.py#L71-L96
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/attention_processor.py#L3572-L3665

Problem:
`LuminaNextDiT2DModel` uses the legacy shared `Attention` plus `LuminaAttnProcessor2_0`, whose processor calls `F.scaled_dot_product_attention` directly and has no `_attention_backend` / `_parallel_config`.

Impact:
`set_attention_backend()` and the `attention_backend(...)` context manager cannot actually select Flash/Sage/xFormers/context-parallel dispatch for Lumina attention.

Reproduction:
```python
from diffusers import LuminaNextDiT2DModel

model = LuminaNextDiT2DModel(sample_size=4, hidden_size=24, num_layers=1, num_attention_heads=3, num_kv_heads=1, multiple_of=16, learn_sigma=False, cross_attention_dim=32)
processors = [m.processor for m in model.modules() if hasattr(m, "processor")]
print([type(p).__name__ for p in processors])
print([hasattr(p, "_attention_backend") for p in processors])
model.set_attention_backend("native")
print([getattr(p, "_attention_backend", None) for p in processors])
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_flux.py#L75-L125

Suggested fix:
Port Lumina attention to the current processor pattern: define the processor in the model file, add `_attention_backend` and `_parallel_config`, and call `dispatch_attention_fn(...)` instead of `F.scaled_dot_product_attention(...)`.

## Issue 7: Lumina quantization docs use the wrong component classes

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/docs/source/en/api/pipelines/lumina.md#L88-L107

Problem:
The docs load the Lumina text encoder with `T5EncoderModel` and the transformer with `Transformer2DModel`. The checkpoint config is Gemma text encoder plus `LuminaNextDiT2DModel`.

Impact:
Users following the quantization docs hit config/class errors before inference.

Reproduction:
```python
from diffusers import LuminaNextDiT2DModel, Transformer2DModel
from transformers import AutoConfig, T5EncoderModel

repo = "Alpha-VLLM/Lumina-Next-SFT-diffusers"
text_config = AutoConfig.from_pretrained(repo, subfolder="text_encoder")
print(text_config.model_type) # gemma
try:
T5EncoderModel(text_config)
except Exception as e:
print(type(e).__name__, str(e).splitlines()[0])

config = LuminaNextDiT2DModel.load_config(repo, subfolder="transformer")
try:
Transformer2DModel.from_config(config)
except Exception as e:
print(type(e).__name__, str(e).splitlines()[0])
```

Relevant precedent:
The pipeline type hints and tests use Gemma classes and `LuminaNextDiT2DModel`.
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/pipelines/lumina/test_lumina_nextdit.py#L6-L13

Suggested fix:
```python
from diffusers import BitsAndBytesConfig as DiffusersBitsAndBytesConfig, LuminaNextDiT2DModel, LuminaPipeline
from transformers import BitsAndBytesConfig as TransformersBitsAndBytesConfig, GemmaModel

text_encoder_8bit = GemmaModel.from_pretrained(..., quantization_config=...)
transformer_8bit = LuminaNextDiT2DModel.from_pretrained(..., quantization_config=...)
```

Start with the affected entry points in src/diffusers/pipelines/lumina/pipeline_lumina.py, src/diffusers/models/transformers/lumina_nextdit2d.py, src/diffusers/models/attention_processor.py, and docs/source/en/api/pipelines/lumina.md. Run the targeted Lumina model, pipeline, and slow pipeline tests after resolving the reported collection limitation; done means the seven listed behaviors are corrected and the relevant tests and documentation pass.

索引モデルが issue の本文から書いたものです。