marl / marl/openl3

error when initializing the model

Open
#96 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
604
Forks
66
PR merge metrics
No merged PRs in 30d

Description

Hi! I'm just following the tutorial doing these:
```
import openl3
model = openl3.models.load_audio_embedding_model(
input_repr="mel128",
content_type='music',
embedding_size=512,
)
```
And an error pops out:
```
Traceback (most recent call last):
File "", line 1, in
File "/ext3/miniconda3/lib/python3.11/site-packages/openl3/models.py", line 134, in load_audio_embedding_model
return load_audio_embedding_model_from_path(model_path, input_repr, embedding_size, frontend=frontend)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ext3/miniconda3/lib/python3.11/site-packages/openl3/models.py", line 166, in load_audio_embedding_model_from_path
m = AUDIO_MODELS[input_repr](include_frontend=frontend == 'kapre')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ext3/miniconda3/lib/python3.11/site-packages/openl3/models.py", line 413, in _construct_mel128_audio_network
spec = __fix_kapre_spec(get_melspectrogram_layer)(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ext3/miniconda3/lib/python3.11/site-packages/openl3/models.py", line 44, in get_spectrogram
seq = func(*a, return_decibel=False, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ext3/miniconda3/lib/python3.11/site-packages/kapre/composed.py", line 261, in get_melspectrogram_layer
return Sequential(layers, name=name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/models/sequential.py", line 74, in __init__
self._maybe_rebuild()
File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/models/sequential.py", line 139, in _maybe_rebuild
self.build(input_shape)
File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/layers/layer.py", line 222, in build_wrapper
original_build_method(*args, **kwargs)
File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/models/sequential.py", line 180, in build
x = layer(x)
^^^^^^^^
File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/ext3/miniconda3/lib/python3.11/site-packages/optree/ops.py", line 594, in tree_map
return treespec.unflatten(map(func, *flat_args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Exception encountered when calling STFT.call().

Could not automatically infer the output shape / dtype of 'stft_3' (of type STFT). Either the `STFT.call()` method is incorrect, or you need to implement the `STFT.compute_output_spec() / compute_output_shape()` method. Error encountered:

Invalid dtype: complex64

Arguments received by STFT.call():
• args=('',)
• kwargs=
```
Any idea what I did wrong here? Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproducer in the issue and trace load_audio_embedding_model through openl3/models.py, especially the mel128 construction around _construct_mel128_audio_network. Then inspect the kapre spectrogram/STFT path shown in the traceback and verify the relevant dependency compatibility. Done means the tutorial model initialization completes without the reported RuntimeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
keras, python, tensorflow
Domain
audio-video-rtc, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.