esphome / esphome/feature-requests
Use Espressif ESP-SR speech recognition framework to allow local on-device Wake Word on ESP32?
- Dominant language
- No language data
- Stars
- 450
- Forks
- 29
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the problem you have/What new integration you would like**
Use "WakeNet Wake Word Engine" from ESP-SR (Espressif Speech Recognition framework) to enable on-device Wake Word on ESP32.
https://www.cnx-software.com/2023/07/17/espressif-esp-sr-enables-on-device-speech-recognition-framework-on-esp32-s3-and-esp32-wisocs/
ESP-SR framework v1.0 was first released on December 17, 2021 and the ESP-SR v1.20 update was introduced in March of 2023
https://github.com/espressif/esp-sr
https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3/audio_front_end/README.html
Espressif wake word engine WakeNet is specially designed to provide a high performance and low memory footprint wake word detection algorithm for users, which enables devices always listen to wake words, such as “Alexa”, “Hi,lexin” and “Hi,ESP”. You can refer to Model loading method to build your project.
Currently, Espressif has not only provided an official wake word "Hi,Lexin","Hi,ESP" to the public for free, but also allows customized wake words. For details on how to customize your own wake words, please see Espressif Speech Wake Words Customization Process.
https://docs.espressif.com/projects/esp-sr/en/latest/esp32s3/wake_word_engine/README.html
**Please describe your use case for this integration and alternatives you've tried:**
Local on-device Wake Word with ESPHome on ESP32 (ESP32-S3 or better) as part of Home Assistant's year of Voice effort.
https://www.home-assistant.io/blog/2022/12/20/year-of-voice/
**Additional context**
That Project utilizes the ESP32-S3 microcontroller and [INMP441 Omnidirectional Microphone](https://www.amazon.com/AITRIP-Omnidirectional-Microphone-Precision-Interface/dp/B092HWW4RS?crid=16MFK65X7FU1H&keywords=inmp441&qid=1684301162&sprefix=inmp44,aps,222&sr=8-3&th=1&linkCode=sl1&tag=thatproject05-20&linkId=20af90df92ac5bb8c0de5daa5a9f4e7e&language=en_US&ref_=as_li_ss_tl) to enable voice command functionality by leveraging the Espressif Speech Recognition framework, the system can recognize wake-up words like "Hey Siri", and "Ok Google" and execute specific actions associated with those commands. The voice user interface provides a convenient means of interaction in screenless environments and can be applied to a wide range of projects. The implementation involves configuring the ESP32-S3 and integrating the INMP441 microphone to capture audio input, processing the speech recognition using the framework, and linking recognized commands to predefined actions. Overall, this project enables users to control various tasks and functions by speaking voice commands, enhancing usability in scenarios where traditional screen-based interfaces are not available.
https://twitter.com/EspressifSystem/status/1680232917476446208
https://www.youtube.com/watch?v=3XbnzfBjmZk&ab_channel=ThatProject
https://www.youtube.com/watch?v=qq2FRv0lCPw&ab_channel=ThatProject
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked ESP-SR project and its WakeNet and model-loading documentation. Investigate the ESP32-S3 and INMP441 setup described in the issue, then determine how local wake-word detection would connect to ESPHome actions. Done means a defined integration path for on-device wake words and recognized commands.
Written by the indexing model from the issue text.
Assessment
- Domain
- embedded-iot
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100