huggingface / huggingface/diffusers

`guidance_scale=1.0` no longer disables CFG in LTX-2 pipelines

Open
#14,649 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

### Describe the bug

`LTX2Pipeline.do_classifier_free_guidance` is:

```python
return (self._guidance_scale > 1.0) or (self._audio_guidance_scale > 1.0)
```

and since #14447 ("Ltx 2.5", merged 2026-08-11) the `__call__` defaults changed:

| default | before #14447 | after |
|---|---|---|
| `audio_guidance_scale` | `None` (follows `guidance_scale`) | **`7.0`** |
| `stg_scale` | `0.0` | **`1.0`** (with `spatio_temporal_guidance_blocks=[28]`) |
| `guidance_rescale` | `0.0` | `0.7` |

Consequence: every distilled recipe that passes only `guidance_scale=1.0` — including the
**LTX-2.3 distilled examples currently in the docs** (`docs/source/en/api/pipelines/ltx2.md`,
"Distilled checkpoint generation" and the condition-pipeline distilled example) and third-party
scripts written before #14447 — now silently runs:

- classifier-free guidance driven by `audio_guidance_scale=7.0`, with the **negative prompt live**
and a **doubled transformer batch every step**, and
- spatio-temporal guidance (`stg_scale=1.0`, block 28 perturbed).

The distilled checkpoints are trained for CFG=1 ("runs in 8 steps with CFG = 1"), so this both
doubles compute and degrades output. The 2.5-distilled examples added in #14447 compensate by
passing `audio_guidance_scale=1.0` explicitly; the pre-existing 2.3-distilled examples were not
updated and are now broken.

Adding `audio_guidance_scale=1.0, stg_scale=0.0, audio_stg_scale=0.0` restores the intended
unguided behavior.

### Proposed fix

Since LTX-2 has no use case where one modality is guided while the other is disabled, gate CFG on
**both** scales, making `guidance_scale=1.0` a master off-switch again while the recommended
base-model settings (`3.0` video / `7.0` audio) still enable it:

```python
return (self._guidance_scale > 1.0) and (self._audio_guidance_scale > 1.0)
```

Applied to `LTX2Pipeline`, `LTX2ConditionPipeline`, `LTX2ImageToVideoPipeline`, and the IC-LoRA
variant. No config used anywhere in the repo changes behavior except the broken one: docs use
`3.0/7.0` (base — CFG stays on) and `1.0/1.0` (2.5 distilled — off either way); tests use
`1.0/1.0`; the only `1.0/7.0` occurrences are the stale distilled examples this fixes.

Complementary follow-ups, in whatever combination maintainers prefer:

- Update the LTX-2.3 distilled doc examples to also pass `stg_scale=0.0, audio_stg_scale=0.0`
(STG has a separate gate and stays on by default via `spatio_temporal_guidance_blocks=[28]`;
whether STG-on is intended for distilled checkpoints is worth clarifying — the 2.5-distilled
examples leave it on).
- Longer-term: gate guidance on a registered `is_distilled` config flag, as
`Flux2KleinPipeline` already does (it warns and skips CFG for distilled checkpoints).

I have the `and` change ready as a commit and can open a PR if the approach is acceptable.

### Affected repos in the wild

Repos matching LTX-2 for diffusers snippets passing `guidance_scale=1.0`
without `audio_guidance_scale`; these ship broken examples today:

- `diffusers/LTX-2.3-Distilled-Diffusers` (official diffusers-org conversion)
- `rootonchair/LTX-2.3-Distilled-v1.1-Diffusers`
- `rootonchair/LTX-2-19b-distilled` (also referenced by the docs' distilled example)
- `lite-infer/LTX-2.3-Distilled-v1.1-Diffusers-nunchaku-lite-int4-bnb4-text-encoder`
- `lite-infer/LTX-2.3-Distilled-v1.1-Diffusers-nunchaku-lite-nvfp4-bnb4-text-encoder`
- `Lightricks/LTX-2`

Post-#14447 model cards (`Lightricks/LTX-2.5`, `Lightricks/LTX-2.5-Diffusers`) already pass
`audio_guidance_scale=1.0` and are unaffected — further evidence the older examples were simply
left behind by the defaults change.

### Reproduction

```python
import torch
from diffusers import LTX2Pipeline
from diffusers.pipelines.ltx2.utils import DEFAULT_NEGATIVE_PROMPT, DISTILLED_SIGMA_VALUES

pipe = LTX2Pipeline.from_pretrained("rootonchair/LTX-2.3-Distilled-v1.1-Diffusers", torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

out = pipe(
prompt="A flowing river in a forest at golden hour, gentle wind in the leaves.",
negative_prompt=DEFAULT_NEGATIVE_PROMPT,
width=768, height=512, num_frames=121, frame_rate=24.0,
num_inference_steps=8, sigmas=DISTILLED_SIGMA_VALUES,
guidance_scale=1.0, # user believes guidance is off
generator=torch.Generator("cuda").manual_seed(42),
)
print(pipe.do_classifier_free_guidance) # True (audio_guidance_scale defaulted to 7.0)
print(pipe.do_spatio_temporal_guidance) # True (stg_scale defaulted to 1.0)
```

### Logs

```shell

```

### System Info

- diffusers: main (reproduced at 0.41.0.dev0)
- torch 2.11.0+cu128, transformers 5.16.1

### Who can help?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start at LTX2Pipeline.do_classifier_free_guidance and compare the same entry point in LTX2ConditionPipeline, LTX2ImageToVideoPipeline, and the IC-LoRA variant. Review docs/source/en/api/pipelines/ltx2.md, especially the LTX-2.3 distilled examples, and verify the existing guidance-scale cases still behave as intended. Done means distilled examples no longer enable unintended guidance while base-model settings remain enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
documentation, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.