lumina model/pipeline review

# `lumina` model/pipeline review

Commit tested: `0f1abc4ae8b0eb2a3b40e82a310507281144c423`

Review performed against the repository review rules.

Reviewed target files, top-level/lazy exports, dummy exports, docs, fast/slow tests, dtype/device handling, offload-related config, and attention processor behavior. Fast model and pipeline tests exist, and a slow Lumina pipeline test exists. Targeted pytest collection was attempted with `.venv`, but the local Torch build is missing `torch._C._distributed_c10d`, so shared test mixins fail during collection before Lumina tests run.

Duplicate search status: focused `gh` and GitHub connector searches found no exact duplicates for the Lumina findings. Related but not exact duplicates: https://github.com/huggingface/diffusers/pull/10827, https://github.com/huggingface/diffusers/issues/13613, https://github.com/huggingface/diffusers/pull/11368.

## Issue 1: Deprecated alias is exported but cannot be constructed

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L940-L950
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/__init__.py#L624-L625

Problem:
`LuminaText2ImgPipeline` is still publicly exported, but its constructor calls `deprecate(..., "0.34", ...)`. Current version is `0.38.0.dev0`, so construction raises `ValueError` instead of warning.

Impact:
Users can import the backwards-compatible alias, but any path that instantiates it fails immediately.

Reproduction:
```python
from diffusers import LuminaText2ImgPipeline

LuminaText2ImgPipeline(None, None, None, None, None)
```

Relevant precedent:
Related rename PR, but it does not remove/fix the current exported alias failure: https://github.com/huggingface/diffusers/pull/10827

Suggested fix:
```python
# Either remove the alias from pipeline_lumina.py, lazy exports, top-level exports,
# and dummy objects, or bump the deprecation target to a future version if keeping it.
deprecate(
"diffusers.pipelines.lumina.pipeline_lumina.LuminaText2ImgPipeline",
"1.0.0",
deprecation_message,
)
```

## Issue 2: VAE scale factor is hardcoded

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L196-L197

Problem:
`vae_scale_factor` is hardcoded to `8` instead of derived from `vae.config.block_out_channels`.

Impact:
Custom/tiny VAEs compute wrong latent sizes, default image sizes, and input divisibility checks.

Reproduction:
```python
from diffusers import AutoencoderKL, FlowMatchEulerDiscreteScheduler, LuminaNextDiT2DModel, LuminaPipeline

transformer = LuminaNextDiT2DModel(sample_size=4, hidden_size=24, num_layers=1, num_attention_heads=3, num_kv_heads=1, multiple_of=16, learn_sigma=False, cross_attention_dim=32)
vae = AutoencoderKL(block_out_channels=(32, 64), down_block_types=("DownEncoderBlock2D", "DownEncoderBlock2D"), up_block_types=("UpDecoderBlock2D", "UpDecoderBlock2D"), latent_channels=4)
pipe = LuminaPipeline(transformer, FlowMatchEulerDiscreteScheduler(), vae, None, None)

print(pipe.vae_scale_factor) # 8
print(2 ** (len(vae.config.block_out_channels) - 1)) # 2
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/flux/pipeline_flux.py#L198-L212

Suggested fix:
```python
self.vae_scale_factor = 2 ** (len(self.vae.config.block_out_channels) - 1) if getattr(self, "vae", None) else 8
self.image_processor = VaeImageProcessor(vae_scale_factor=self.vae_scale_factor)
```

## Issue 3: `max_sequence_length` is ignored

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L206-L222
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L782-L794

Problem:
`__call__` accepts `max_sequence_length`, but `encode_prompt` only captures it in `**kwargs`, and `_get_gemma_prompt_embeds` always uses `self.max_sequence_length`.

Impact:
Users cannot shorten or adjust prompt tokenization through the documented pipeline argument.

Reproduction:
```python
import torch
from types import SimpleNamespace
from diffusers import LuminaPipeline

class Tok:
def __call__(self, prompt, **kw):
n = kw.get("max_length") or 11
return SimpleNamespace(input_ids=torch.zeros(len(prompt), n, dtype=torch.long), attention_mask=torch.ones(len(prompt), n))
def batch_decode(self, ids): return [""]

class Enc(torch.nn.Module):
dtype = torch.float32
def forward(self, input_ids, **kw):
h = torch.zeros(input_ids.shape[0], input_ids.shape[1], 8)
return SimpleNamespace(hidden_states=[h, h, h])

pipe = LuminaPipeline.__new__(LuminaPipeline)
pipe.max_sequence_length, pipe.tokenizer, pipe.text_encoder, pipe.transformer = 256, Tok(), Enc(), None
embeds, mask, *_ = pipe.encode_prompt("x", do_classifier_free_guidance=False, device=torch.device("cpu"), max_sequence_length=7)
print(embeds.shape, mask.shape) # torch.Size([1, 256, 8]) torch.Size([1, 256])
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina2/pipeline_lumina2.py#L192-L215

Suggested fix:
```python
def _get_gemma_prompt_embeds(..., max_sequence_length: int = 256):
...
max_length=max_sequence_length
...
removed_text = self.tokenizer.batch_decode(untruncated_ids[:, max_sequence_length - 1 : -1])

# and forward max_sequence_length from encode_prompt into _get_gemma_prompt_embeds
```

## Issue 4: Prompt conditioning expansion is wrong for multiple images

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L252-L257
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L310-L369

Problem:
Generated prompt masks use `repeat(num_images_per_prompt, 1)`, which orders masks as `[p1, p2, p1, p2]` while embeddings are `[p1, p1, p2, p2]`. Also, when users pass precomputed prompt/negative embeddings, `encode_prompt` does not expand them for `num_images_per_prompt`.

Impact:
Batched prompts with different mask lengths can pair embeddings with the wrong masks, and precomputed-embedding workflows fail or mis-broadcast when generating multiple images per prompt.

Reproduction:
```python
import torch
from diffusers import LuminaPipeline

pipe = LuminaPipeline.__new__(LuminaPipeline)
pe = torch.zeros(1, 5, 8)
pm = torch.ones(1, 5, dtype=torch.long)
ne = torch.ones(1, 5, 8)
nm = torch.ones(1, 5, dtype=torch.long)

out = pipe.encode_prompt(
prompt=None, do_classifier_free_guidance=True, num_images_per_prompt=2,
device=torch.device("cpu"), prompt_embeds=pe, prompt_attention_mask=pm,
negative_prompt_embeds=ne, negative_prompt_attention_mask=nm,
)
print(out[0].shape, out[2].shape) # both still batch 1; expected batch 2 each
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/pixart_alpha/pipeline_pixart_alpha.py#L390-L431

Suggested fix:
```python
bs, seq_len, _ = prompt_embeds.shape
prompt_embeds = prompt_embeds.repeat(1, num_images_per_prompt, 1).view(bs * num_images_per_prompt, seq_len, -1)
prompt_attention_mask = prompt_attention_mask.repeat(1, num_images_per_prompt).view(bs * num_images_per_prompt, -1)

if do_classifier_free_guidance:
bs, seq_len, _ = negative_prompt_embeds.shape
negative_prompt_embeds = negative_prompt_embeds.repeat(1, num_images_per_prompt, 1).view(bs * num_images_per_prompt, seq_len, -1)
negative_prompt_attention_mask = negative_prompt_attention_mask.repeat(1, num_images_per_prompt).view(bs * num_images_per_prompt, -1)
```

## Issue 5: Provided latents are not cast to the requested dtype

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/lumina/pipeline_lumina.py#L597-L615

Problem:
`prepare_latents` moves user-provided latents to the device but leaves dtype unchanged.

Impact:
Supplying float32 latents to a bf16/fp16 pipeline can feed mismatched activations into lower-precision transformer weights.

Reproduction:
```python
import torch
from diffusers import LuminaPipeline

pipe = LuminaPipeline.__new__(LuminaPipeline)
pipe.vae_scale_factor = 8
latents = torch.ones(1, 4, 2, 2, dtype=torch.float32)
out = pipe.prepare_latents(1, 4, 16, 16, torch.bfloat16, torch.device("cpu"), None, latents)
print(out.dtype) # torch.float32
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3.py#L633-L646

Suggested fix:
```python
else:
latents = latents.to(device=device, dtype=dtype)
```

## Issue 6: Lumina attention bypasses the attention backend dispatcher

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/lumina_nextdit2d.py#L71-L96
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/attention_processor.py#L3572-L3665

Problem:
`LuminaNextDiT2DModel` uses the legacy shared `Attention` plus `LuminaAttnProcessor2_0`, whose processor calls `F.scaled_dot_product_attention` directly and has no `_attention_backend` / `_parallel_config`.

Impact:
`set_attention_backend()` and the `attention_backend(...)` context manager cannot actually select Flash/Sage/xFormers/context-parallel dispatch for Lumina attention.

Reproduction:
```python
from diffusers import LuminaNextDiT2DModel

model = LuminaNextDiT2DModel(sample_size=4, hidden_size=24, num_layers=1, num_attention_heads=3, num_kv_heads=1, multiple_of=16, learn_sigma=False, cross_attention_dim=32)
processors = [m.processor for m in model.modules() if hasattr(m, "processor")]
print([type(p).__name__ for p in processors])
print([hasattr(p, "_attention_backend") for p in processors])
model.set_attention_backend("native")
print([getattr(p, "_attention_backend", None) for p in processors])
```

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_flux.py#L75-L125

Suggested fix:
Port Lumina attention to the current processor pattern: define the processor in the model file, add `_attention_backend` and `_parallel_config`, and call `dispatch_attention_fn(...)` instead of `F.scaled_dot_product_attention(...)`.

## Issue 7: Lumina quantization docs use the wrong component classes

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/docs/source/en/api/pipelines/lumina.md#L88-L107

Problem:
The docs load the Lumina text encoder with `T5EncoderModel` and the transformer with `Transformer2DModel`. The checkpoint config is Gemma text encoder plus `LuminaNextDiT2DModel`.

Impact:
Users following the quantization docs hit config/class errors before inference.

Reproduction:
```python
from diffusers import LuminaNextDiT2DModel, Transformer2DModel
from transformers import AutoConfig, T5EncoderModel

repo = "Alpha-VLLM/Lumina-Next-SFT-diffusers"
text_config = AutoConfig.from_pretrained(repo, subfolder="text_encoder")
print(text_config.model_type) # gemma
try:
T5EncoderModel(text_config)
except Exception as e:
print(type(e).__name__, str(e).splitlines()[0])

config = LuminaNextDiT2DModel.load_config(repo, subfolder="transformer")
try:
Transformer2DModel.from_config(config)
except Exception as e:
print(type(e).__name__, str(e).splitlines()[0])
```

Relevant precedent:
The pipeline type hints and tests use Gemma classes and `LuminaNextDiT2DModel`.
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/pipelines/lumina/test_lumina_nextdit.py#L6-L13

Suggested fix:
```python
from diffusers import BitsAndBytesConfig as DiffusersBitsAndBytesConfig, LuminaNextDiT2DModel, LuminaPipeline
from transformers import BitsAndBytesConfig as TransformersBitsAndBytesConfig, GemmaModel

text_encoder_8bit = GemmaModel.from_pretrained(..., quantization_config=...)
transformer_8bit = LuminaNextDiT2DModel.from_pretrained(..., quantization_config=...)
```

Bắt đầu với các điểm vào bị ảnh hưởng trong src/diffusers/pipelines/lumina/pipeline_lumina.py, src/diffusers/models/transformers/lumina_nextdit2d.py, src/diffusers/models/attention_processor.py và docs/source/en/api/pipelines/lumina.md. Chạy các bài kiểm thử Lumina model, pipeline và slow pipeline được nhắm mục tiêu sau khi giải quyết hạn chế collection đã được báo cáo; hoàn tất có nghĩa là bảy hành vi được liệt kê đã được sửa và các bài kiểm thử cùng tài liệu liên quan đều đạt.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.