modelscope / modelscope/DiffSynth-Studio
LTX2.3 data process when input_audio process error
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
Traceback (most recent call last):
File "/root/autodl-tmp/DiffSynth-Studio/examples/ltx2/model_training/train.py", line 179, in
launcher_map[args.task](accelerator, dataset, model, model_logger, args=args)
File "/root/autodl-tmp/DiffSynth-Studio/diffsynth/diffusion/runner.py", line 65, in launch_data_process_task
for data_id, data in enumerate(tqdm(dataloader)):
File "/root/miniconda3/lib/python3.12/site-packages/tqdm/std.py", line 1181, in iter
for obj in iterable:
File "/root/miniconda3/lib/python3.12/site-packages/accelerate/data_loader.py", line 567, in iter
current_batch = next(dataloader_iter)
^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 734, in next
data = self._next_data()
^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 790, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "/root/autodl-tmp/DiffSynth-Studio/diffsynth/core/data/unified_dataset.py", line 98, in getitem
data[key] = self.special_operator_mapkey
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/autodl-tmp/DiffSynth-Studio/diffsynth/core/data/operators.py", line 14, in call
data = operator(data)
^^^^^^^^^^^^^^
File "/root/autodl-tmp/DiffSynth-Studio/diffsynth/core/data/operators.py", line 266, in call
waveform, sample_rate = torchaudio.load(data)
^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torchaudio/_backend/utils.py", line 222, in load
return backend.load(uri, frame_offset, num_frames, normalize, channels_first, format, buffer_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torchaudio/_backend/soundfile.py", line 27, in load
return soundfile_backend.load(uri, frame_offset, num_frames, normalize, channels_first, format)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/torchaudio/backend/soundfile_backend.py", line 221, in load
with soundfile.SoundFile(filepath, "r") as file:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/soundfile.py", line 690, in init
self._file = self._open(file, mode_int, closefd)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/lib/python3.12/site-packages/soundfile.py", line 1265, in _open
raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name))
soundfile.LibsndfileError: Error opening 'data/ltx2.3/LTX-doggy-style/0001.mp4': Format not recognised.
Traceback (most recent call last):
File "/root/miniconda3/bin/accelerate", line 8, in
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/ltx2/model_training/train.py and diffsynth/diffusion/runner.py, then follow UnifiedDataset.getitem and the operator at diffsynth/core/data/operators.py:266. Reproduce the failure for data/ltx2.3/LTX-doggy-style/0001.mp4 and define completion as input_audio processing no longer aborting on this dataset.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- data-engineering, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100