`hunyuan_video1_5` model/pipeline review

hunyuan_video1_5 model/pipeline review

Commit tested: 0f1abc4ae8b0eb2a3b40e82a310507281144c423

Review performed against the repository review rules in .ai/review-rules.md and referenced rule files.

Local checks:

• Top-level imports for the HunyuanVideo 1.5 model, standard pipelines, and modular pipeline classes passed.
• tests/modular_pipelines/hunyuan_video1_5/test_modular_pipeline_hunyuan_video1_5.py passed with 11 passed, 3 skipped.
• tests/models/transformers/test_models_transformer_hunyuan_1_5.py and tests/pipelines/hunyuan_video1_5/test_hunyuan_1_5.py could not collect in this local .venv because the installed Torch build lacks torch._C._distributed_c10d, which is imported by shared training test utilities.

Duplicate search performed before filing:

• Searched Issues and PRs for hunyuan_video1_5, HunyuanVideo15*, affected class/function names, num_videos_per_prompt, VAE batch attention mask, modular glyph regex, slow/image2video coverage, and doc snippet failures.
• Known duplicates are called out in the relevant items below.

Issue 1: VAE attention mask breaks batch > 1

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanvideo15.py#L150-L154

Problem:
HunyuanVideo15AttnBlock passes a (batch, seq, seq) mask to 4D SDPA queries shaped (batch, 1, seq, channels). For batch > 1, PyTorch broadcasts the mask incorrectly and raises.

Duplicate:
This is already covered by PR https://github.com/huggingface/diffusers/pull/13133.

Impact:
Batch generation, num_videos_per_prompt > 1, and the skipped modular batch tests fail at VAE encode/decode time.

Reproduction:

import torch
from diffusers import AutoencoderKLHunyuanVideo15

vae = AutoencoderKLHunyuanVideo15(
    in_channels=3,
    out_channels=3,
    latent_channels=4,
    block_out_channels=(16, 16),
    layers_per_block=1,
    spatial_compression_ratio=4,
    temporal_compression_ratio=2,
    downsample_match_channel=False,
    upsample_match_channel=False,
).eval()

with torch.no_grad():
    vae(torch.randn(2, 3, 9, 16, 16), return_dict=False)

Relevant precedent:
PR #13133 applies the same fix and cites the original implementation.

Suggested fix:

x = nn.functional.scaled_dot_product_attention(query, key, value, attn_mask=attention_mask.unsqueeze(1))

Issue 2: T2V num_videos_per_prompt > 1 does not repeat zero image embeddings

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/hunyuan_video1_5/pipeline_hunyuan_video1_5.py#L719-L725
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/modular_pipelines/hunyuan_video1_5/before_denoise.py#L244-L250

Problem:
The T2V path expands latents and text embeddings to batch_size * num_videos_per_prompt, but zero image_embeds are allocated with only batch_size. The transformer then receives mismatched batch dimensions.

Duplicate:
No duplicate issue/PR found.

Impact:
num_videos_per_prompt > 1 fails before denoising finishes. The modular test suite currently skips this exact path.

Reproduction:

import torch
from diffusers import HunyuanVideo15Transformer3DModel

model = HunyuanVideo15Transformer3DModel(
    in_channels=9, out_channels=4, num_attention_heads=2, attention_head_dim=8,
    num_layers=1, num_refiner_layers=1, mlp_ratio=2.0, patch_size=1, patch_size_t=1,
    text_embed_dim=16, text_embed_2_dim=32, image_embed_dim=12,
    rope_axes_dim=(2, 2, 4), target_size=16, task_type="t2v",
).eval()

with torch.no_grad():
    model(
        hidden_states=torch.randn(2, 9, 1, 2, 4),
        timestep=torch.ones(2),
        encoder_hidden_states=torch.randn(2, 6, 16),
        encoder_attention_mask=torch.ones(2, 6),
        encoder_hidden_states_2=torch.randn(2, 4, 32),
        encoder_attention_mask_2=torch.ones(2, 4),
        image_embeds=torch.zeros(1, 3, 12),
        return_dict=False,
    )

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/hunyuan_video1_5/pipeline_hunyuan_video1_5_image2video.py#L776-L781

Suggested fix:

image_embeds = torch.zeros(
    batch_size * num_videos_per_prompt,
    self.vision_num_semantic_tokens,
    self.vision_states_dim,
    dtype=self.transformer.dtype,
    device=device,
)

For the modular path, use the effective batch size when constructing zero-filled image_embeds in HunyuanVideo15PrepareLatentsStep.

Issue 3: Modular glyph extraction drops curly-quoted text

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/modular_pipelines/hunyuan_video1_5/encoders.py#L46-L48

Problem:
The modular regex checks straight quotes twice and omits the curly quote branch used by the standard pipeline.

Duplicate:
This is already covered by PR https://github.com/huggingface/diffusers/pull/13523.

Impact:
Prompts using “...” lose glyph text conditioning in the modular pipeline, so text rendering behavior diverges from the standard pipeline.

Reproduction:

from diffusers.modular_pipelines.hunyuan_video1_5.encoders import extract_glyph_texts as modular_extract
from diffusers.pipelines.hunyuan_video1_5.pipeline_hunyuan_video1_5 import extract_glyph_texts as standard_extract

prompt = 'A sign says “HELLO”.'
assert standard_extract(prompt) == 'Text "HELLO". '
assert modular_extract(prompt) is None

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/pipelines/hunyuan_video1_5/pipeline_hunyuan_video1_5.py#L93-L103

Suggested fix:

pattern = r"\"(.*?)\"|“(.*?)”"

Issue 4: Modular blocks bypass declared IO contracts

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/modular_pipelines/hunyuan_video1_5/encoders.py#L171-L202
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/src/diffusers/modular_pipelines/hunyuan_video1_5/encoders.py#L307-L390

Problem:
HunyuanVideo15TextEncoderStep writes batch_size with state.set() without declaring it as an output, and HunyuanVideo15VaeEncoderStep mutates image the same way. The modular rules require writing through declared outputs via block state.

Duplicate:
No duplicate issue/PR found.

Impact:
The generated block interface/docs are incomplete, standalone block reuse is harder, and downstream dependencies can be hidden from modular validation.

Reproduction:

import inspect
from diffusers.modular_pipelines.hunyuan_video1_5.encoders import (
    HunyuanVideo15TextEncoderStep,
    HunyuanVideo15VaeEncoderStep,
)

text_step = HunyuanVideo15TextEncoderStep()
vae_step = HunyuanVideo15VaeEncoderStep()

assert "batch_size" not in [p.name for p in text_step.intermediate_outputs]
assert 'state.set("batch_size"' in inspect.getsource(text_step.__call__)
assert "image" not in [p.name for p in vae_step.intermediate_outputs]
assert 'state.set("image"' in inspect.getsource(vae_step.__call__)

Relevant precedent:
The modular review rule says not to call state.set() inside a block and to declare every written output.

Suggested fix:
Declare OutputParam("batch_size", type_hint=int) and set block_state.batch_size = batch_size. Avoid mutating image, or declare it as an output if the resized/cropped image is part of the public block contract.

Issue 5: Missing VAE/I2V/slow coverage and skipped batch coverage

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/pipelines/hunyuan_video1_5/test_hunyuan_1_5.py#L42-L195
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/modular_pipelines/hunyuan_video1_5/test_modular_pipeline_hunyuan_video1_5.py#L49-L79
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/tests/models/autoencoders/test_models_autoencoder_hunyuan_video.py#L20-L32

Problem:
There is no fast test class for HunyuanVideo15ImageToVideoPipeline, no AutoencoderKLHunyuanVideo15 model test, and no slow tests for the HunyuanVideo 1.5 family. The modular test uses a contributor repo (akshan-main/...) instead of hf-internal-testing/... and skips num_videos_per_prompt plus batch consistency.

Duplicate:
No duplicate issue/PR found.

Impact:
The failures above are not caught in CI, and slow-test coverage is absent.

Reproduction:

from pathlib import Path

checks = {
    "i2v_pipeline_fast": "HunyuanVideo15ImageToVideoPipeline" in Path("tests/pipelines/hunyuan_video1_5/test_hunyuan_1_5.py").read_text(),
    "vae15_model_fast": "AutoencoderKLHunyuanVideo15Tests" in Path("tests/models/autoencoders/test_models_autoencoder_hunyuan_video.py").read_text(),
    "target_slow": "@slow" in Path("tests/pipelines/hunyuan_video1_5/test_hunyuan_1_5.py").read_text(),
    "internal_tiny_model": "hf-internal-testing/" in Path("tests/modular_pipelines/hunyuan_video1_5/test_modular_pipeline_hunyuan_video1_5.py").read_text(),
}
print(checks)
assert checks == {key: True for key in checks}

Relevant precedent:
Existing video families such as tests/pipelines/hunyuan_video/test_hunyuan_image2video.py keep I2V coverage separate from T2V coverage.

Suggested fix:
Add fast tests for HunyuanVideo15ImageToVideoPipeline and AutoencoderKLHunyuanVideo15, add slow tests for published T2V/I2V checkpoints, move the modular tiny fixture to hf-internal-testing/, and unskip batch/num_videos_per_prompt tests after Issues 1 and 2 are fixed.

Issue 6: HunyuanVideo15 docs contain broken loading snippets

Affected code:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/docs/source/en/api/models/hunyuan_video15_transformer_3d.md#L18-L22
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/docs/source/en/api/pipelines/hunyuan_video15.md#L35-L43

Problem:
The transformer snippet is a Python syntax error because it misses a comma before subfolder. The pipeline snippet also imports unused AutoModel and uses an unqualified model id despite the surrounding text saying the examples use hunyuanvideo-community.

Duplicate:
No duplicate issue/PR found.

Impact:
Users copying the docs hit immediate syntax or loading errors.

Reproduction:

import ast

snippet = '''
from diffusers import HunyuanVideo15Transformer3DModel

transformer = HunyuanVideo15Transformer3DModel.from_pretrained("hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v" subfolder="transformer", torch_dtype=torch.bfloat16)
'''
ast.parse(snippet)

Relevant precedent:
https://github.com/huggingface/diffusers/blob/0f1abc4ae8b0eb2a3b40e82a310507281144c423/docs/source/en/api/models/autoencoder_kl_hunyuan_video15.md#L18-L24

Suggested fix:

import torch
from diffusers import HunyuanVideo15Transformer3DModel

transformer = HunyuanVideo15Transformer3DModel.from_pretrained(
    "hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v",
    subfolder="transformer",
    torch_dtype=torch.bfloat16,
)

Start with the affected HunyuanVideo 1.5 pipeline, modular block, encoder, and test files named in the issue, then run the modular test file and the listed reproductions. Separate items already covered by PRs #13133 and #13523 from the remaining work; done means the remaining defects have regression coverage, the missing model and pipeline tests are added, and both documentation snippets are valid.