pytorch / pytorch/executorch

QNN: wav2letter doesn't export

Open
#7,634 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

module: qnn partner: qualcomm triaged
Dominant language
Python
Stars
5k
Forks
1.2k
Avg merge
2d 10h
Merged PRs (30d)
581

Description

🐛 Describe the bug
python -m examples.qualcomm.scripts.wav2letter -b cmake-out -m SM8450 --compile_only -p ./states_fused.pth

fails to export. See stacktrace:

QNN_SDK_ROOT=/home/guangyang/fbsource/third-party/qualcomm/qnn/qnn-2.26
Traceback (most recent call last):
  File "/data/users/guangyang/executorch/examples/qualcomm/scripts/wav2letter.py", line 220, in <module>
    main(args)
  File "/data/users/guangyang/executorch/examples/qualcomm/scripts/wav2letter.py", line 131, in main
    inputs, targets, input_list = get_dataset(
  File "/data/users/guangyang/executorch/examples/qualcomm/scripts/wav2letter.py", line 69, in get_dataset
    for wave, label in data_loader:
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 708, in __next__
    data = self._next_data()
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 764, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 52, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/site-packages/torchaudio/datasets/librispeech.py", line 170, in __getitem__
    waveform = _load_waveform(self._archive, metadata[0], metadata[1])
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/site-packages/torchaudio/datasets/utils.py", line 51, in _load_waveform
    waveform, sample_rate = torchaudio.load(path)
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/site-packages/torchaudio/_backend/utils.py", line 204, in load
    backend = dispatcher(uri, format, backend)
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/site-packages/torchaudio/_backend/utils.py", line 116, in dispatcher
    raise RuntimeError(f"Couldn't find appropriate backend to handle uri {uri} and format {format}.")
RuntimeError: Couldn't find appropriate backend to handle uri ./wav2letter/LibriSpeech/test-clean/2094/142345/2094-142345-0042.flac and format None.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/guangyang/.conda/envs/executorch/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/data/users/guangyang/executorch/examples/qualcomm/scripts/wav2letter.py", line 226, in <module>
    raise Exception(e)
Exception: Couldn't find appropriate backend to handle uri ./wav2letter/LibriSpeech/test-clean/2094/142345/2094-142345-0042.flac and format None.

where states_fused.pth is downloaded from https://github.com/nipponjo/wav2letter-ctc-pytorch/tree/main?tab=readme-ov-file#wav2letter-ctc-pytorch

cc: @cccclai @cbilgin

Versions

latest main

cc @cccclai @winskuo-quic @shewu-quic

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/qualcomm/scripts/wav2letter.py, especially get_dataset and the main entry point, then inspect the torchaudio.load failure for the LibriSpeech .flac path. Re-run the provided wav2letter command while checking the dataset and audio-loading setup; done means the script exports successfully with the stated model and options.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
embedded-iot, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.