Automattic / Automattic/pocket-casts-ios

Spatial Audio: figure out what's required to support spatial audio in EffectsPlayer

Open
#29 0 comments 0 reactions 0 assignees View on GitHub
[Type] Enhancement playback
Dominant language
Swift
Stars
1.8k
Forks
197
Avg merge
17h 20m
Merged PRs (30d)
103

Description

When a podcast is played through `DefaultPlayer` (eg: streamed with trim silence off) and you have the app set to play spatial audio, the effect works out of the box.

However when you play a podcast through `EffectsPlayer` (eg: downloaded, or trim silence downloaded on the fly content) it doesn't work.

That seems to be what this video is saying as well: https://developer.apple.com/videos/play/wwdc2021/10265/

That support for it is out of the box in `AVPlayer` (the playback engine behind `DefaultPlayer`) and enabled by default, but that you need to do some work to get it to work with `AVAudioEngine` (the playback engine behind `EffectsPlayer`)

Apparently [AVSampleBufferAudioRenderer](https://developer.apple.com/documentation/avfoundation/avsamplebufferaudiorenderer) supports it so that might be one thing to look into. There's an entire sample project for it here:
https://developer.apple.com/documentation/avfaudio/audio_engine/playing_custom_audio_with_your_own_player

Contributor guide

Open the contributing guide

Research direction

Start by comparing the DefaultPlayer path using AVPlayer with the EffectsPlayer path using AVAudioEngine, focusing on why spatial audio works for one but not the other. Read the linked WWDC session and the AVSampleBufferAudioRenderer and custom-player sample documentation. Done means the required implementation approach and scope for spatial audio in EffectsPlayer are established.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
audio-video-rtc
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.