modelscope / modelscope/DiffSynth-Studio

LTX2.3 data process when input_audio process error

Open
#1,366 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

Traceback (most recent call last):
File "/root/autodl-tmp/DiffSynth-Studio/examples/ltx2/model_training/train.py", line 179, in
launcher_map[args.task](accelerator, dataset, model, model_logger, args=args)
File "/root/autodl-tmp/DiffSynth-Studio/diffsynth/diffusion/runner.py", line 65, in launch_data_process_task
for data_id, data in enumerate(tqdm(dataloader)):
File "/root/miniconda3/lib/python3.12/site-packages/tqdm/std.py", line 1181, in iter
for obj in iterable:
File "/root/miniconda3/lib/python3.12/site-packages/accelerate/data_loader.py", line 567, in iter
current_batch = next(dataloader_iter)
^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 734, in next
data = self._next_data()
^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 790, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "/root/autodl-tmp/DiffSynth-Studio/diffsynth/core/data/unified_dataset.py", line 98, in getitem
data[key] = self.special_operator_mapkey
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/autodl-tmp/DiffSynth-Studio/diffsynth/core/data/operators.py", line 14, in call
data = operator(data)
^^^^^^^^^^^^^^
File "/root/autodl-tmp/DiffSynth-Studio/diffsynth/core/data/operators.py", line 266, in call
waveform, sample_rate = torchaudio.load(data)
^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torchaudio/_backend/utils.py", line 222, in load
return backend.load(uri, frame_offset, num_frames, normalize, channels_first, format, buffer_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torchaudio/_backend/soundfile.py", line 27, in load
return soundfile_backend.load(uri, frame_offset, num_frames, normalize, channels_first, format)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torchaudio/backend/soundfile_backend.py", line 221, in load
with soundfile.SoundFile(filepath, "r") as file
:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/soundfile.py", line 690, in init
self._file = self._open(file, mode_int, closefd)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/soundfile.py", line 1265, in _open
raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name))
soundfile.LibsndfileError: Error opening 'data/ltx2.3/LTX-doggy-style/0001.mp4': Format not recognised.
Traceback (most recent call last):
File "/root/miniconda3/bin/accelerate", line 8, in

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/ltx2/model_training/train.py and diffsynth/diffusion/runner.py, then follow UnifiedDataset.getitem and the operator at diffsynth/core/data/operators.py:266. Reproduce the failure for data/ltx2.3/LTX-doggy-style/0001.mp4 and define completion as input_audio processing no longer aborting on this dataset.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
data-engineering, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.