esphome / esphome/feature-requests

Audio Signal processing / Audio analyzer component

Open
#2,444 2 comments 6 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
450
Forks
29
PR merge metrics
No merged PRs in 30d

Description

**Describe the problem you have/What new integration you would like**
A VU meter and VAD (voice activity detection) available to the i2s_audio microphone and eventually other means of signal processing for audio. Available within in the Arduino framework for ESP32.

**Please describe your use case for this integration and alternatives you've tried:**
The current state of the 'voice_assistant' is sub-par. VAD and silence detection rely on the ESP-ADF which isn't free software and are implemented inside of the 'voice_assistant' component. I'd like a seperate component available to do audio signal processing. A VU meter / silence detection / voice activity detection and we could also implement [clap detection](https://github.com/esphome/feature-requests/issues/795), whistle detection and stuff like that. Maybe later on noise suppression and wake-word detection. This could all seperated into one component.

**Additional context**

Is there an empty component / similar component / boilerplate code available to get me started with a component like that? I've read the documentation on components but there is a steep learning curve until you get the boilerplate-code usable.

I'm not aware of any good signal processing libraries. [ArduinoSound](https://github.com/arduino-libraries/ArduinoSound/) has some simple examples available, and there is [arduino-audio-tools](https://github.com/pschatzmann/arduino-audio-tools/).

And can we use the ESP-DSP? It's available both for the ESP-IDF and as part of the ESP32 Arduino framework. It contains things like FFT. Or is this too custom because it only works for ESP32?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the component documentation and examining the existing i2s_audio and voice_assistant components. Compare the ArduinoSound, arduino-audio-tools, and ESP-DSP options mentioned in the request, then determine the initial scope for a separate audio signal-processing component. Done would require an agreed, reusable component boundary and a defined first capability such as VU metering or VAD.

Written by the indexing model from the issue text.

Assessment

Tech stack
arduino
Domain
audio-video-rtc, embedded-iot
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.