huggingface / huggingface/diffusers
LTX2Pipeline: dynamic timestep shift `mu` is constant, ignores the actual sequence length
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
In LTX2Pipeline.__call__ (src/diffusers/pipelines/ltx2/pipeline_ltx2.py), mu is computed with max_image_seq_len passed as the first argument of calculate_shift — the image_seq_len parameter:
mu = calculate_shift(
self.scheduler.config.get("max_image_seq_len", 4096), # <-- this is image_seq_len; should be the sample's real sequence length
self.scheduler.config.get("base_image_seq_len", 1024),
self.scheduler.config.get("max_image_seq_len", 4096),
self.scheduler.config.get("base_shift", 0.95),
self.scheduler.config.get("max_shift", 2.05),
)
calculate_shift(image_seq_len, base, max, base_shift, max_shift) returns image_seq_len * m + b. With image_seq_len == max_seq_len, that reduces to max_shift, so mu is constant regardless of resolution or frame count — use_dynamic_shifting=True has no effect.
image_seq_len should be the sample's packed sequence length, latent_num_frames * latent_height * latent_width — which is already computed a few lines above (and even present as the commented line # video_sequence_length = latent_num_frames * latent_height * latent_width). This matches the Flux/SD3 pipelines and the LTX reference (math.prod(latent.shape[2:])).
Fix: pass latent_num_frames * latent_height * latent_width as the first argument.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/diffusers/pipelines/ltx2/pipeline_ltx2.py, inside LTX2Pipeline.call, and read the nearby latent dimension calculations and calculate_shift call. Confirm how the packed sequence length is derived and compare the analogous Flux/SD3 pipeline logic. Done means dynamic shifting uses the actual latent sequence length and varies with resolution or frame count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100