bria_fibo model/pipeline review

オープン

#13,618 2026年4月29日に作成コメント 0 件リアクション 0 件担当者 0 名 GitHub で見る

主要言語: Python
スター: 34.5k
フォーク: 7.3k
平均マージ: 3日 3時間
マージ済み PR（30日）: 91

説明

# `bria_fibo` model/pipeline review

Commit tested: `0f1abc4ae8b0eb2a3b40e82a310507281144c423`

Review performed against the repository review rules.

Duplicate search: checked GitHub issues/PRs for `bria_fibo`, `BriaFibo`, `FIBO`, affected class names, and failure modes. Found integration/refactor PRs `#12545`, `#12688`, `#12731`, `#12930`, `#13341`; no duplicate for the issues below. Public top-level imports succeeded.

Test note: standalone reproductions ran with `.venv`. Full target pytest collection failed before running tests because this Windows torch build lacks `torch._C._distributed_c10d`.

## Issue 1: `prompt_embeds` is public but unusable

Problem:
Both pipelines expose `prompt_embeds`, but `encode_prompt()` only defines `prompt_layers` when it encodes `prompt` itself. Passing precomputed embeddings raises `UnboundLocalError`. `negative_prompt_embeds` is also not honored because negative embeddings are always recomputed when `guidance_scale > 1`.

Impact:
Users cannot use documented precomputed embedding workflows, prompt weighting, cached text encoder outputs, or callback-modified embeddings reliably.

Reproduction:
```python
import torch
from types import SimpleNamespace
from diffusers import BriaFiboPipeline

pipe = BriaFiboPipeline.__new__(BriaFiboPipeline)
pipe.transformer = SimpleNamespace(dtype=torch.float32)
pipe.text_encoder = SimpleNamespace(dtype=torch.float32)

pipe.encode_prompt(
prompt=None,
prompt_embeds=torch.zeros(1, 1, 64),
guidance_scale=1.0,
device=torch.device("cpu"),
)
```

Relevant precedent:
Flux validates all required precomputed conditioning inputs together:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/flux/pipeline_flux.py#L494-L500

Suggested fix:
```python
if prompt_embeds is not None and prompt_layers is None:
raise ValueError("`prompt_embeds` requires precomputed `prompt_layers`, or pass `prompt` instead.")
```
Better: add public `prompt_layers` / `negative_prompt_layers` inputs and honor `negative_prompt_embeds` instead of recomputing it.

## Issue 2: Custom `timesteps` are ignored

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo.py#L466-L480
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo.py#L674-L680
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo_edit.py#L626-L642
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo_edit.py#L892-L898

Problem:
`timesteps` is accepted and documented, but both pipelines call `retrieve_timesteps(..., timesteps=None, sigmas=sigmas, ...)`.

Impact:
Users requesting a custom timestep schedule silently get the default schedule.

Reproduction:
```python
import inspect
from diffusers import BriaFiboPipeline, BriaFiboEditPipeline

for cls in (BriaFiboPipeline, BriaFiboEditPipeline):
source = inspect.getsource(cls.__call__)
assert "timesteps=None" not in source, f"{cls.__name__} drops custom timesteps"
```

Relevant precedent:
The shared helper supports `timesteps`:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/flux/pipeline_flux.py#L88-L129

Suggested fix:
```python
sigmas = None if timesteps is not None else np.linspace(1.0, 1 / num_inference_steps, num_inference_steps)
timesteps, num_inference_steps = retrieve_timesteps(
self.scheduler,
num_inference_steps=num_inference_steps,
device=device,
timesteps=timesteps,
sigmas=sigmas,
mu=mu,
)
```

## Issue 3: Tensor images crash in `BriaFiboEditPipeline`

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo_edit.py#L809-L812

Problem:
The tensor-image path reads `self.latent_channels`, but that attribute is never defined.

Impact:
`image=torch.Tensor(...)` is accepted by validation and documented typing, but crashes before preprocessing.

Reproduction:
```python
import torch
from diffusers import BriaFiboEditPipeline

pipe = BriaFiboEditPipeline.__new__(BriaFiboEditPipeline)
image = torch.zeros(1, 3, 32, 32)

if image is not None and not (isinstance(image, torch.Tensor) and image.size(1) == pipe.latent_channels):
pass
```

Relevant precedent:
N/A.

Suggested fix:
```python
if image is not None:
image = self.image_processor.resize(image, height, width)
image = self.image_processor.preprocess(image, height, width)
```
If latent image input is intended, define the latent channel count from `self.transformer.config.in_channels` and validate it explicitly.

## Issue 4: Multiple generated images get malformed output shape

Problem:
Each per-sample `postprocess(..., output_type="np")` returns shape `(1, H, W, C)`, then the pipeline uses `np.stack`, producing `(N, 1, H, W, C)` instead of `(N, H, W, C)`. PIL output becomes nested lists.

Impact:
`num_images_per_prompt > 1` returns an incompatible output structure.

Reproduction:
```python
import numpy as np

per_sample = [np.zeros((1, 32, 32, 3)), np.zeros((1, 32, 32, 3))]
print(np.stack(per_sample, axis=0).shape) # (2, 1, 32, 32, 3)
```

Relevant precedent:
Bria decodes and postprocesses the batch directly:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria/pipeline_bria.py#L730-L733

Suggested fix:
```python
curr_image = self.image_processor.postprocess(curr_image.squeeze(dim=2), output_type=output_type)
if output_type == "np":
image.append(curr_image[0])
else:
image.extend(curr_image)
...
if output_type == "np":
image = np.stack(image, axis=0)
```

## Issue 5: Edit pipeline does not duplicate image latents for `num_images_per_prompt`

Problem:
`prepare_image_latents()` receives `batch_size * num_images_per_prompt`, but the encoded image batch remains at the original image batch size before reshape.

Impact:
`BriaFiboEditPipeline(..., image=..., num_images_per_prompt=2)` fails with an invalid reshape. The fast tests skip batching, so this is not covered.

Reproduction:
```python
import torch
from diffusers import BriaFiboEditPipeline

latents = torch.zeros(1, 16, 2, 2)
BriaFiboEditPipeline._pack_latents_no_patch(
latents=latents,
batch_size=2,
num_channels_latents=16,
height=2,
width=2,
)
```

Relevant precedent:
N/A.

Suggested fix:
```python
repeat_by = batch_size // image_latents_bchw.shape[0]
image_latents_bchw = image_latents_bchw.repeat_interleave(repeat_by, dim=0)
```
Also unskip/add batch tests for edit.

## Issue 6: `guidance_embeds=True` cannot construct the transformer

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_bria_fibo.py#L415-L426
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_bria_fibo.py#L472-L473
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_bria_fibo.py#L558-L559

Problem:
`BriaFiboTimestepProjEmbeddings` requires `time_theta`, but `guidance_embed` is constructed without it. The forward path also uses `if guidance:` on a tensor.

Impact:
Any config/checkpoint with `guidance_embeds=True` fails during model construction, and the forward branch would be ambiguous for multi-element tensors after construction is fixed.

Reproduction:
```python
from diffusers import BriaFiboTransformer2DModel

BriaFiboTransformer2DModel(
patch_size=1,
in_channels=16,
num_layers=1,
num_single_layers=1,
attention_head_dim=8,
num_attention_heads=2,
joint_attention_dim=64,
text_encoder_dim=32,
axes_dims_rope=[0, 4, 4],
guidance_embeds=True,
)
```

Relevant precedent:
Flux checks `guidance is None`, not tensor truthiness:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_flux.py#L682-L690

Suggested fix:
```python
if guidance_embeds:
self.guidance_embed = BriaFiboTimestepProjEmbeddings(
embedding_dim=self.inner_dim,
time_theta=time_theta,
)

...
if guidance is not None:
temb += self.guidance_embed(guidance, dtype=hidden_states.dtype)
```

## Issue 7: Dense additive attention masks disable flash/sage attention

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo.py#L647-L653
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo_edit.py#L852-L871
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_bria_fibo.py#L111-L118

Problem:
The pipelines convert padding masks into dense `(B, 1, L, L)` additive float masks. This is only padding information and can be represented as a bool key mask, but dense masks hard-fail for flash-attn and sage backends.

Impact:
Users selecting optimized attention backends hit avoidable runtime failures.

Reproduction:
```python
import torch
from diffusers.pipelines.bria_fibo.pipeline_bria_fibo import BriaFiboPipeline
from diffusers.models.attention_dispatch import _flash_attention

mask = torch.tensor([[1, 1, 0, 1]], dtype=torch.float32)
dense_mask = BriaFiboPipeline._prepare_attention_mask(mask).unsqueeze(1)
q = k = v = torch.randn(1, 4, 2, 8)

_flash_attention(q, k, v, attn_mask=dense_mask)
```

Relevant precedent:
QwenImage builds a bool joint mask instead:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_qwenimage.py#L946-L952

Suggested fix:
```python
attention_mask = torch.cat([prompt_attention_mask, latent_attention_mask], dim=1).to(torch.bool)
attention_mask = attention_mask[:, None, None, :]
self._joint_attention_kwargs["attention_mask"] = attention_mask
```

## Issue 8: VAE scale factor is hardcoded instead of read from config

Problem:
Both pipelines set `self.vae_scale_factor = 16` even though `AutoencoderKLWan` stores `scale_factor_spatial` in config.

Impact:
Custom or future Fibo-compatible VAEs with a different spatial scale serialize/load correctly but produce wrong latent sizes in the pipeline.

Reproduction:
```python
from diffusers import AutoencoderKLWan, BriaFiboPipeline

vae = AutoencoderKLWan(base_dim=8, decoder_base_dim=8, num_res_blocks=1, z_dim=4, dim_mult=[1], temperal_downsample=[])
pipe = BriaFiboPipeline(transformer=None, scheduler=None, vae=vae, text_encoder=None, tokenizer=None)

print(vae.config.scale_factor_spatial) # 8
print(pipe.vae_scale_factor) # 16
```

Relevant precedent:
Wan reads the VAE scale factor from config:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/wan/pipeline_wan.py#L154-L156

Suggested fix:
```python
self.vae_scale_factor = self.vae.config.scale_factor_spatial if getattr(self, "vae", None) else 16
```

## Issue 9: Transformer is missing `_no_split_modules`

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_bria_fibo.py#L430-L445

Problem:
The model enables gradient checkpointing but does not declare `_no_split_modules` for its transformer blocks.

Impact:
`device_map` / offload placement can split residual attention blocks across devices, unlike comparable transformer integrations.

Reproduction:
```python
from diffusers import BriaFiboTransformer2DModel

print(getattr(BriaFiboTransformer2DModel, "_no_split_modules", None))
```

Relevant precedent:
Flux declares both block classes:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/transformers/transformer_flux.py#L565-L567

Suggested fix:
```python
_no_split_modules = ["BriaFiboTransformerBlock", "BriaFiboSingleTransformerBlock"]
```

## Issue 10: Edit example docstring is stale and not runnable

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo_edit.py#L53-L86

Problem:
The file contains `# TODO: Update example docstring`, imports `ModularPipeline`, then uses undefined `ModularPipelineBlocks`.

Impact:
Generated docs include a broken example, and the TODO violates the review rule against ephemeral PR-context comments.

Reproduction:
```python
namespace = {}
exec(
"from diffusers.modular_pipelines import ModularPipeline\n"
"ModularPipelineBlocks.from_pretrained('briaai/FIBO-VLM-prompt-to-JSON')",
namespace,
)
```

Relevant precedent:
The text-to-image Fibo example uses `ModularPipeline.from_pretrained`:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/bria_fibo/pipeline_bria_fibo.py#L51-L57

Suggested fix:
```python
vlm_pipe = ModularPipeline.from_pretrained("briaai/FIBO-VLM-prompt-to-JSON", trust_remote_code=True)
```
Also remove the TODO and verify the model id casing.

## Issue 11: Slow tests are missing

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/models/transformers/test_models_transformer_bria_fibo.py#L29-L37
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/pipelines/bria_fibo/test_pipeline_bria_fibo.py#L39-L47
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/pipelines/bria_fibo_edit/test_pipeline_bria_fibo_edit.py#L40-L47

Problem:
Fast model and pipeline tests exist, but there are no `@slow` tests for `BriaFiboPipeline` or `BriaFiboEditPipeline`.

Impact:
The gated real checkpoints are never exercised for loading, dtype/offload behavior, output shape, JSON prompt handling, or edit image/mask behavior.

Reproduction:
```python
from pathlib import Path

paths = [
Path("tests/models/transformers/test_models_transformer_bria_fibo.py"),
Path("tests/pipelines/bria_fibo/test_pipeline_bria_fibo.py"),
Path("tests/pipelines/bria_fibo_edit/test_pipeline_bria_fibo_edit.py"),
]
print({str(path): "@slow" in path.read_text() for path in paths})
```

Relevant precedent:
Bria has a slow pipeline test:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/pipelines/bria/test_pipeline_bria.py#L241-L245

Suggested fix:
Add gated slow smoke tests for `briaai/FIBO` and `briaai/Fibo-Edit`, using `torch_dtype=torch.bfloat16`, `enable_model_cpu_offload()`, a short schedule, deterministic seed/generator, and expected output shape/value slices.

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Start with the affected files under src/diffusers/pipelines/bria_fibo/ and src/diffusers/models/transformers/transformer_bria_fibo.py, then run the standalone reproductions in the issue. Treat the eight findings as separate regressions and add targeted coverage for embeddings, timesteps, tensor images, batching, guidance, attention masks, and configuration-driven scaling. Done means the documented inputs work without the reported exceptions or malformed outputs, with regression tests passing where collection is available.

索引モデルが issue の本文から書いたものです。

評価

技術スタック: python, pytorch
領域: computer-vision, machine-learning, testing
issue の種類: バグ
難易度: 5/5
見積もり時間: 1週間以上
活発さ: 静か
明瞭さ: おおむね明確
初心者へのやさしさ: 25/100

bria_fibo model/pipeline review

説明

コントリビューションガイド

調査の方向性

評価

新しい issue をメールで受け取る