MiniMax-H3: `references` argument ignored and documented multi-GPU example fails with CUDA device mismatch

### Describe the bug

## Environment

- OS: CentOS 7
- GPUs: 8 × NVIDIA A100 80GB
- Python: 3.11
- PyTorch: 2.6.0+cu124
- CUDA Driver: 550.xx (CUDA 12.4)
- Diffusers branches tested:
- `minimax-h3`
- `minimax-h3-refactor`

The same issues occur on both branches.

---

## Description

I tried to reproduce the MiniMax-H3 Reference-to-Video example from the documentation:

https://github.com/huggingface/diffusers/blob/minimax-h3/docs/source/en/api/pipelines/minimax_h3.md

I encountered two independent issues.

---

## Issue 1: `references` argument is ignored

Calling the pipeline with

```python
state = pipe(
prompt="...",
references=[
MiniMaxH3Reference(image=subject),
],
)
```

produces

```text
UserWarning:
Unexpected input 'dict_keys(['references'])' provided.
This input will be ignored.
```

As a result, all reference inputs are ignored by the pipeline.

This happens on both `minimax-h3` and `minimax-h3-refactor`.

---

## Issue 2: Official multi-GPU example fails with device mismatch

Following the official documentation, I place the Qwen3-VL text encoder on GPU1 while keeping the transformer on GPU0.

```python
import torch

from diffusers import ComponentsManager, ModularPipeline
from transformers import Qwen3VLForConditionalGeneration

manager = ComponentsManager()

pipe = ModularPipeline.from_pretrained(
"MiniMaxAI/MiniMax-H3",
components_manager=manager,
)

text_encoder = Qwen3VLForConditionalGeneration.from_pretrained(
"MiniMaxAI/MiniMax-H3",
subfolder="text_encoder",
dtype=torch.bfloat16,
device_map={"": "cuda:1"},
)

pipe.update_components(
text_encoder=text_encoder,
)

pipe.load_components(dtype=torch.bfloat16)

pipe.transformer.to("cuda:0")
pipe.vae.to("cuda:0")
pipe.audio_vae.to("cuda:0")
```

The pipeline initializes successfully, but fails during denoising with

```text
RuntimeError:
Expected all tensors to be on the same device,
but found at least two devices, cuda:1 and cuda:0
```

The traceback points to

```python
transformer_minimax_h3.py

rotary_emb = self.rope(position_ids)

freqs = position_ids.unsqueeze(-1) * self.inv_freq.view(...)
```

Apparently,

- `position_ids` is located on `cuda:1`
- `self.inv_freq` is located on `cuda:0`

No automatic activation transfer happens between the text encoder and the transformer.

The full traceback ends with

```text
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!
```

---

## Expected behavior

According to the documentation,

1. `references` should be accepted by the pipeline.
2. The documented two-GPU loading strategy should work without additional user modifications.
3. Intermediate tensors produced by the text encoder should be transferred automatically (or otherwise handled) before entering the transformer.

---

## Questions

1. Is the current documentation ahead of the implementation?
2. Is the `references` argument currently supported in `ModularPipeline`?
3. Is the documented multi-GPU example expected to work, or is this feature still under development?
4. Is there an officially recommended way to run MiniMax-H3 on two 80GB GPUs without CPU offloading?

Thanks for the great work on MiniMax-H3 support. I'd be happy to test any proposed fixes or provide additional logs if needed.

### Reproduction

import torch

from diffusers import ComponentsManager, ModularPipeline
from transformers import Qwen3VLForConditionalGeneration

from diffusers.modular_pipelines.minimax_h3 import MiniMaxH3Reference
from diffusers.utils import load_image, load_video
from diffusers.utils.export_utils import encode_video

# -------------------------
# Pipeline
# -------------------------

manager = ComponentsManager()

pipe = ModularPipeline.from_pretrained(
"MiniMaxAI/MiniMax-H3",
components_manager=manager,
)

print("Loading Qwen3-VL on GPU1...")

text_encoder = Qwen3VLForConditionalGeneration.from_pretrained(
"MiniMaxAI/MiniMax-H3",
subfolder="text_encoder",
dtype=torch.bfloat16,
#device_map={"": "cuda:1"},
)

pipe.load_components(dtype=torch.bfloat16)

manager.enable_auto_cpu_offload(
device="cuda",
memory_reserve_margin="12GB",
)

pipe.update_components(
text_encoder=text_encoder,
)

print("Loading remaining components...")

pipe.load_components(dtype=torch.bfloat16)

print("Moving transformer to GPU0...")

pipe.transformer.to("cuda:0")

pipe.vae.to("cuda:0")

pipe.audio_vae.to("cuda:0")

print("Pipeline Ready.")

subject = load_image(
"extracted_images/img_row8_col3_12.png"
)

product = load_image(
"extracted_images/img_row23_col3_48.png"
)

state = pipe(
prompt="subject_definitions: is the elderly female host in , sitting in a cozy streaming setup with a dignified and wise appearance. is the health supplement in , a premium wellness product with trustworthy packaging and clear health-focused branding. summary: [reference generation] The target video shows at her streaming desk in a fixed camera livestream, carefully picking up and presenting to the audience with a caring and authoritative manner, emphasizing the product's health benefits. retention_analysis: : fully_preserved - the host's dignified elderly appearance and wise demeanor are retained throughout. : fully_preserved - the health product's trustworthy packaging and branding are retained. detailed_description: The target video is a fixed-camera livestream shot in a warm and authoritative health style with soft, warm lighting and a comfortable, home-like background. The camera remains stationary throughout. A medium shot shows , the elderly woman, sitting at her streaming desk with a composed and dignified expression. She carefully picks up from the display area. She holds the product steadily at a clear viewing height, presenting its label side to the camera. Her hands cradle the product with care, and her expression conveys deep personal endorsement. She nods gently, reinforcing the trustworthiness of her recommendation. overall_soundscape: Warm, quiet room tone with a gentle, calming atmosphere. non_diegetic_music: N/A",
references=[
MiniMaxH3Reference(image=subject),
MiniMaxH3Reference(image=product),
],
num_frames=124,
)

encode_video(
state["videos"][0],
fps=24,
output_path="result.mp4",
audio=state["audio"][0],
audio_sample_rate=state["sampling_rate"],
)

### Logs

```shell

```

### System Info

- 🤗 Diffusers version: 0.40.0.dev0
- Platform: Linux-4.18.0-2.6.8.kwai.x86_64-x86_64-with-glibc2.17
- Running on Google Colab?: No
- Python version: 3.11.0
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Huggingface_hub version: 1.26.0
- Transformers version: 5.14.1
- Accelerate version: 1.14.0
- PEFT version: not installed
- Safetensors version: 0.8.0
- xFormers version: not installed
- Accelerator: NVIDIA A100-SXM4-80GB, 81920 MiB
NVIDIA A100-SXM4-80GB, 81920 MiB
NVIDIA A100-SXM4-80GB, 81920 MiB
NVIDIA A100-SXM4-80GB, 81920 MiB
NVIDIA A100-SXM4-80GB, 81920 MiB
NVIDIA A100-SXM4-80GB, 81920 MiB
NVIDIA A100-SXM4-80GB, 81920 MiB
NVIDIA A100-SXM4-80GB, 81920 MiB
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help?

_No response_

Start with docs/source/en/api/pipelines/minimax_h3.md and the MiniMax-H3 pipeline entry points used by ModularPipeline, then inspect transformer_minimax_h3.py around the rotary-embedding call. Run the supplied reproduction to confirm the ignored references argument and the cuda:1/cuda:0 mismatch. Done means references are accepted and the documented two-GPU example completes without a device error, with regression coverage for both cases.

Written by the indexing model from the issue text.