AOSSIE-Org / AOSSIE-Org/Resonate

iOS: Integrate JUCE DSP with Native Audio Capture and LiveKit Publishing

Open
#686 2 comments 0 reactions 1 assignee Claimed by @uju09 View on GitHub
enhancement
Dominant language
Dart
Stars
344
Forks
350
PR merge metrics
No merged PRs in 30d

Description

### 🚀 The feature

On iOS, replace LiveKit’s default microphone capture with a custom audio pipeline that:
• Captures microphone audio using AVAudioEngine
• Processes audio buffers using the shared JUCE DSP engine (via Objective-C++)
• Publishes processed audio buffers to LiveKit AudioTrack

The existing LiveKit transport layer remains unchanged.

### Motivation, pitch

To enable real-time, on-device voice processing on iOS, microphone audio must be captured natively and processed before being handed to LiveKit.

Using JUCE via Objective-C++ allows us to reuse the same DSP logic implemented for Android, ensuring consistent audio behavior across platforms. This approach maintains privacy, avoids server-side audio modification, and aligns with iOS real-time audio best practices.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.