software-mansion / software-mansion/react-native-audio-api

Android: platform AEC doesn't cancel app's own playback unless the OUTPUT stream uses Usage::VoiceCommunication

Open Beginner friendly
#1,254 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

android bug
Dominant language
C++
Stars
839
Forks
92
Avg merge
4d 15h
Merged PRs (30d)
27

Description

Feature Request

When building a full-duplex voice assistant (mic capture + TTS playback at the same
time), enabling androidInputPreset: 'voiceCommunication' on the AudioRecorder alone
was not enough to get working acoustic echo cancellation on Android. Our STT kept
transcribing the assistant's own TTS playback.

The missing piece: the output stream (AudioPlayer.cpp) is opened without any
Usage, which defaults to USAGE_MEDIA. On many devices, the platform AEC chain
engaged by the VOICE_COMMUNICATION input preset only references the
voice-communication output path and media-stream playback leaks into the microphone
uncancelled.

Adding two lines to the output stream builder fixed it completely:

// AudioPlayer.cpp — openAudioStream()
builder.setSharingMode(SharingMode::Exclusive)
    ->setFormat(AudioFormat::Float)
    // ...existing config...
    ->setUsage(Usage::VoiceCommunication)   // ← added
    ->setContentType(ContentType::Speech)  // ← added
    // ...
Environment
  • react-native-audio-api 1.0.0-nightly-f8b57cd-20260825
  • React Native 0.86.2 / Expo SDK 57, physical Android device (loudspeaker, no headphones)
  • Capture: AudioRecorder with { androidInputPreset: 'voiceCommunication', iosVoiceProcessing: true }
  • Playback: AudioContext playing TTS PCM buffers
  • Use case: streaming STT (websocket ASR) while speaking TTS replies (barge-in style interaction)
Symptom before the patch
  • Speech-to-text transcribed the assistant's own spoken replies as if they were user input
  • Token-overlap heuristics and LLM-based intent filtering reduced but never eliminated
    the self-echo turns
  • Verified the input preset reaches Oboe correctly (setInputPreset(VoiceCommunication)
    present in openAudioStream()), so the gap was on the output/reference side
After the patch
  • Self-echo eliminated at DSP level (zero phantom transcripts during playback)
  • Real barge-in (user talking over the reply) works reliably
  • No audio-routing regressions on our test device: playback still comes out of the
    loudspeaker as before
Suggestions
  1. Library behavior: consider applying Usage::VoiceCommunication (+ ContentType::Speech)
    to the output stream automatically when an active recorder uses
    androidInputPreset: 'voiceCommunication' or expose an option for it.
  2. Docs (next → AudioRecorder): the full-duplex section recommends
    androidInputPreset: 'voiceCommunication' + iosVoiceProcessing: true, but doesn't
    mention that the playback side must also sit on the voice-communication path for
    Android AEC to reference it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Inspect AudioPlayer.cpp's openAudioStream() alongside AudioRecorder's voiceCommunication handling. Verify full-duplex capture and playback on a physical Android loudspeaker device, then implement the agreed output-stream behavior or document the required configuration in the next AudioRecorder docs. Done means platform AEC removes self-echo without changing loudspeaker routing.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, cpp, react-native
Domain
audio-video-rtc, mobile
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.