error with querying audio embeddings using API
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 213
- PR merge metrics
- No merged PRs in 30d
Description
Summary
I am trying to extract audio features for the Clotho dataset evaluation examples. However, when I try model.get_audio_embedding_from_filelist and model.get_audio_embedding_from_data I get two different errors.
Code For Reproducing
Downloaded Clotho dataset from https://zenodo.org/record/3490684
import laion_clap
from glob import glob
import ipdb
from tqdm import tqdm
import librosa
model = laion_clap.CLAP_Module(enable_fusion=False)
model.load_ckpt()
print(model)
audio_embeddings = []
clotho_eval_path = '/clotho/development/'
audio_files = glob(f'{clotho_eval_path}*.wav')
for audio_file in tqdm(audio_files):
try:
audio_data, _ = librosa.load(audio_file, sr=48000) # sample rate should be 48000
audio_data = audio_data.reshape(1, -1) #Make it (1,T) or (N,T)
audio_embed = model.get_audio_embedding_from_data(x = audio_data, use_tensor=False) # ERROR
except Exception as e:
print("ERROR1:", e)
try:
audio_embed = model.get_audio_embedding_from_filelist(x = audio_file, use_tensor=False) #ALSO ERRROR
except Exception as e:
print("ERROR2:", e)
audio_embeddings.append(audio_embed)
And the errors I am getting:
ERROR1: GET was unable to find an engine to execute this computation
ERROR2: /anaconda3/envs/MPD2/lib/python3.8/site-packages/laion_clap/hook.py:137: UserWarning: PySoundFile failed. Trying audioread instead.
audio_waveform, _ = librosa.load(f, sr=48000)
/anaconda3/envs/MPD2/lib/python3.8/site-packages/librosa/core/audio.py:184: FutureWarning: librosa.core.audio.__audioread_load
Deprecated as of librosa version 0.10.0.
It will be removed in librosa version 1.0.
y, sr_native = __audioread_load(path, offset, duration, dtype)
IsADirectoryError: [Errno 21] Is a directory: '/'
Requirement Versions
librosa 0.10.0
torchlibrosa 0.1.0
transformers 4.30.0
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided Clotho reproduction and inspect the get_audio_embedding_from_data and get_audio_embedding_from_filelist entry points. Compare their expected inputs with the reshaped librosa output and audio_file value; done means the documented audio inputs produce embeddings without either reported error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100