LAION-AI / LAION-AI/emotion-annotations
Annotate_audio.py not outputting transcription.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 112
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
Thanks for creating this great project.
The annotate_audio.py is working well to give emotion annotations, but does not output an accompanying transcription of the speech. This may be my mis-understanding of your use of the word 'transcription'. Your Example Output suggests caption contains 'transcribed text' but actually is a high level description of the text. It would be worth improving naming conventions or adding transcription of spoken text to avoid further confusion.
Thanks,
Caspar
E.g my_audio_file.json:
"caption": "AA medium-quality recording of a male speaker describing a painting. The speaker sounds calm and informative, with a slightly nostalgic tone. The recording quality is decent, with no noticeable background noise.",
There are no obvious error messages although in the console. Although perhaps these are relevant.
Transcription using a multilingual Whisper will default to language detection followed by transcription instead of translation to English. This might be a breaking change for your use case. If you want to instead always translate your audio to English, make sure to pass `language='en'`. See https://github.com/huggingface/transformers/pull/28687 for more details.
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with annotate_audio.py and the README's Example Output to compare the documented meaning of caption with the generated JSON. Determine whether the project should rename the field or add spoken-text transcription, then verify the chosen behavior against the sample output and the reported Whisper messages.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- audio-video-rtc, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100