element-hq / element-hq/element-call

Audio in Android is not always routed correctly

Open
#4,063 0 comments 0 reactions 0 assignees View on GitHub
O-Occasional S-Minor T-Defect
Dominant language
TypeScript
Stars
996
Forks
213
Avg merge
1d 8h
Merged PRs (30d)
55

Description

### Steps to reproduce

In some devices, as this seems to work mostly fine in others, starting a call or joining one will always result in the media volume being the one used for the call audio, even after setting `AudioManager.mode = AudioManager.MODE_IN_COMMUNICATION`.

### Outcome

#### What did you expect?

Call audio volume to be controlled by the 'call' audio volume rocker.

#### What happened instead?

It is controlled by the media one instead.

---

It seems like in some devices (these were a Xiaomi Mi 9 and Poco F6, both built by Xiaomi and using Android 15) I can never get the audio routing to work correctly. In other devices I've seen it either work mostly fine, with some hiccups, or maybe working fine for joining/starting a call and failing for the other action.

I tried several calls between EW and EXA in these devices and the audio always was controlled by the media volume rocker (either pressing vol+ and vol- keys changed the call volume and this had no effect but manually changing the media volume did, or the buttons directly changed the media volume). The logs seemed to display a different audio API being used by the Android system under the hood:

```
# Not working (Xiaomi):

08:33:52.557 android.x.debug io.element.android.x.debug D PlayerBase::PlayerBase()
08:33:52.564 AidlConversionCppNdk io.element.android.x.debug W aidl2legacy_AudioChannelLayout_audio_channel_mask_t: no legacy output audio_channel_mask_t found for AudioChannelLayout{layoutMask: 16}
08:33:52.572 AudioStrea...nal_Client io.element.android.x.debug D configureDataInformation() original HW burst = 96, minMicros = 2000 => SW burst = 96
08:33:52.572 IsochronousClockModel io.element.android.x.debug D setFramesPerBurst() - mFramesPerBurst = 96 - mBurstPeriodNanos = 2000000
08:33:52.572 AAudioStream io.element.android.x.debug D setState(s#9) from 0 to 2
08:33:52.572 AAudioFlowGraph io.element.android.x.debug D configure() source format = 0x00000005, channels = 2, sample rate = 48000, sink format = 0x00000005, channels = 2, sample rate = 48000, useMonoBlend = 0, audioBalance = 0.000000, useVolumeRamps 0
08:33:52.575 AAudio io.element.android.x.debug I AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#9 ----------------
08:33:52.576 AAudio io.element.android.x.debug D AAudioStream_requestStart(s#9) called --------------
08:33:52.576 AAudioStream io.element.android.x.debug D setState(s#9) from 2 to 3
08:33:53.153 AAudio io.element.android.x.debug D AAudioStream_requestStart(s#11) returned 0 ---------
08:33:53.155 AudioStrea...lay_Client io.element.android.x.debug D callbackLoop() entering >>>>>>>>>>>>>>>
08:33:53.155 AudioStrea...nal_Client io.element.android.x.debug D onEventFromServer - got AAUDIO_SERVICE_EVENT_STARTED
08:33:53.155 AAudioStream io.element.android.x.debug D setState(s#11) from 3 to 4
08:33:53.343 chromium io.element.android.x.debug W [WARNING:services/audio/sync_reader.cc:154] ASR: No room in socket buffer.: Broken pipe (32)
08:33:53.344 AAudio io.element.android.x.debug D AAudioStream_requestStop(s#11) called
08:33:53.347 AudioStrea...lay_Client io.element.android.x.debug D callbackLoop() exiting, result = 144, isActive() = 1 <<<<<<<<<<<<<<
08:33:53.348 AAudioStream io.element.android.x.debug D joinThread_l() pthread_join succeeded
08:33:53.348 IsochronousClockModel io.element.android.x.debug D stop(nanos = 33356313370502) max lateness = 35 micros, DSP stalled 0 times
08:33:53.348 AAudioStream io.element.android.x.debug D setState(s#11) from 4 to 9
08:33:53.353 AudioStrea...nal_Client io.element.android.x.debug D onEventFromServer - got AAUDIO_SERVICE_EVENT_STOPPED
08:33:53.354 AAudioStream io.element.android.x.debug D setState(s#11) from 9 to 10
08:33:58.098 AAudio io.element.android.x.debug D AAudioStream_requestStop(s#10) called
08:33:58.098 AAudioStream io.element.android.x.debug D setState(s#10) from 4 to 9
08:33:58.102 org.matrix.rust.sdk io.element.android.x.debug D elementx: [ElementCall] :2 [Lifecycle] LobbyView Component unmounted | ConsoleMessageLogger.kt:49
08:33:58.111 org.matrix.rust.sdk io.element.android.x.debug D elementx: [ElementCall] :2 [InCallView] START CALL VIEW SCOPE | ConsoleMessageLogger.kt:49

# Working (OnePlus):

08:49:04.820 droid.x.nightly io.element.android.x.nightly D PlayerBase::PlayerBase()
08:49:04.822 droid.x.nightly io.element.android.x.nightly D PlayerBase::PlayerBase()
08:49:04.822 AudioStrea...nal_Client io.element.android.x.nightly D ~AudioStreamInternal() 0xb400007b3f1ccc00 called
08:49:04.822 AAudioStream io.element.android.x.nightly D ~AudioStream(s#1) mPlayerBase strongCount = 1
08:49:04.822 droid.x.nightly io.element.android.x.nightly D PlayerBase::~PlayerBase()
08:49:04.822 AudioStreamTrack io.element.android.x.nightly D open(), request notificationFrames = -8, frameCount = 0
08:49:04.822 AudioTrack io.element.android.x.nightly I set(): type=audio,case=1,attributes=1,content_type=2,usage=1,source=0,flags=0x100
08:49:04.822 AudioTrackExtImpl io.element.android.x.nightly D AudioTrackExtImpl init
08:49:04.822 ExtensionsLoader io.element.android.x.nightly D createInstance(64bit) : createExtendedFactory
08:49:04.823 ExtensionsLoader io.element.android.x.nightly D Opened libSchedAssistExtImpl.so
08:49:04.823 AudioTrack io.element.android.x.nightly D set(): streamType -1, sampleRate 48000, format 0x5, channelMask 0x3, frameCount 0, flags #104, notificationFrames -8, sessionId 0, transferType 1, uid -1, pid -1
08:49:04.823 ListServiceUtils io.element.android.x.nightly D mListServiceUtils::init CallingPid 29664
08:49:04.823 ListServiceUtils io.element.android.x.nightly D mListServiceUtils::init this 0xb400007aba4b6c00
08:49:04.823 AudioTrack io.element.android.x.nightly I set(): 0xb400007aba573200, Create AudioTrackThread, tid = 31149
08:49:04.824 AudioTrackExtImpl io.element.android.x.nightly D doSchedBoost, set audioapp task boost, pid=29664, tid=31082, enable=1, ret=0
08:49:04.833 AudioTrack io.element.android.x.nightly D createTrack state 0 output.outputId=13
08:49:04.834 OplusScrollToTopManager io.element.android.x.nightly D io.element.android.x.nightly/io.element.android.x.MainActivity,This com.android.internal.policy.DecorView{d197215 I.E...... R.....ID 0,0-1440,3216 aid=0 alpha=1.0 viewInfo = }[MainActivity] change focus to false
08:49:04.836 AudioTrack io.element.android.x.nightly I createTrack_l(0): AUDIO_OUTPUT_FLAG_FAST successful; frameCount 0 -> 1536
08:49:04.836 AudioTrack io.element.android.x.nightly I createTrack_l(6563) on outputId(13) : 0xb400007aba573200, mCblk = 0x7d565a9000, mLatency = 93, mAfLatency = 61, frameCount = 1536, mSampleRate = 48000, mFlags = 0x4, mReqFrameCount = 1536, mNotificationFramesAct = 192
08:49:04.837 AudioTrackExtImpl io.element.android.x.nightly D doSchedBoost, set audioapp task boost, pid=29664, tid=31082, enable=0, ret=0
08:49:04.837 AudioTrackExtImpl io.element.android.x.nightly E get fade type, but return null, pid 29664, uid 10357
08:49:04.837 AudioTrack io.element.android.x.nightly D set: pop_optimize mIsPopOptimizeSupport:0, mIsPopOptimizeFlush:0
08:49:04.837 AudioTrack io.element.android.x.nightly D setVolume(6563) left 1.000 right 1.000 , callingPid 29664
08:49:04.838 org.matrix.rust.sdk io.element.android.x.nightly D elementx: [ElementCall] :2 [PostmessageTransport] Sending object to https://appassets.androidplatform.net: {"api":"fromWidget","widgetId":"f6798723-1cb0-4480-8209-c0a5e6d67fd5","requestId":"widgetapi-1782197344837","action":"org.matrix.msc4039.download_file","data":{"content_uri":"mxc://element.io/75d9a551895233d284aff390d2086814a2f7e875"}} |
08:49:04.838 AudioStreamTrack io.element.android.x.nightly D open() flags changed from 0x00000104 to 0x00000004
08:49:04.838 AAudioStream io.element.android.x.nightly D setState(s#2) from 0 to 2
08:49:04.839 AAudio io.element.android.x.nightly I AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#2 ----------------
08:49:04.840 AAudio io.element.android.x.nightly D AAudioStream_requestStart(s#2) called --------------
08:49:04.840 AAudioStream io.element.android.x.nightly D setState(s#2) from 2 to 3
08:49:04.840 AudioTrack io.element.android.x.nightly D start(6563): prior state:STATE_STOPPED output 13 stream 3 session 17665
08:49:04.848 SystemSens...gerExtImpl io.element.android.x.nightly V RegisterListener Game Rotation Vector Non-wakeup type:15 delay:16666us by org.chromium.device.sensors.PlatformSensor@65726fb from io.element.android.x.nightly
08:49:04.853 org.matrix.rust.sdk io.element.android.x.nightly D elementx: [ElementCall] :2 [PostmessageTransport] Sending object to https://appassets.androidplatform.net: {"api":"fromWidget","widgetId":"f6798723-1cb0-4480-8209-c0a5e6d67fd5","requestId":"widgetapi-1782197344852","action":"org.matrix.msc4039.download_file","data":{"content_uri":"mxc://element.io/75d9a551895233d284aff390d2086814a2f7e875"}} |
08:49:04.858 org.matrix.rust.sdk io.element.android.x.nightly W elementx: [ElementCall] :2 Could not send DeviceMute action to widget {"data":{}} |
08:49:04.889 AppOps system_server E attributionTag not declared in manifest of io.element.android.x.nightly
08:49:04.889 AppOps system_server E attributionTag not declared in manifest of io.element.android.x.nightly
08:49:04.890 AAudio io.element.android.x.nightly D AAudioStream_requestStart(s#2) returned 0 ---------
08:49:04.891 AudioRecord io.element.android.x.nightly D getMinFrameCount 1920
08:49:04.891 AudioStreamLegacy io.element.android.x.nightly D onAudioDeviceUpdate() devices 3 => 3
08:49:04.892 AudioEffect io.element.android.x.nightly I queryNumberEffects(): *numEffects = 30, ret = 0
08:49:04.899 AudioRecord io.element.android.x.nightly D getMinFrameCount 1920
08:49:04.900 AudioEffect io.element.android.x.nightly I queryNumberEffects(): *numEffects = 30, ret = 0
08:49:04.905 AudioRecord io.element.android.x.nightly D getMinFrameCount 1920
08:49:04.905 AudioEffect io.element.android.x.nightly I queryNumberEffects(): *numEffects = 30, ret = 0
08:49:04.985 AAudioStream io.element.android.x.nightly D setState(s#2) from 3 to 4
08:49:05.478 LOG_FLOWCTRL io.element.android.x.nightly W ==LOGS OVER PROC QUOTA(300), rows(206) bytes(14031) io.element.android.x.nightly u(0) l(0) t(0) c(0) DROPPED==
08:49:05.482 AudioTrack io.element.android.x.nightly D createTrack state 0 output.outputId=13
08:49:05.494 AudioTrack io.element.android.x.nightly I createTrack_l(0): AUDIO_OUTPUT_FLAG_FAST successful; frameCount 0 -> 1536
08:49:05.495 AudioTrack io.element.android.x.nightly I createTrack_l(6568) on outputId(13) : 0xb400007a8e251400, mCblk = 0x7d6e22d000, mLatency = 93, mAfLatency = 61, frameCount = 1536, mSampleRate = 48000, mFlags = 0x4, mReqFrameCount = 1536, mNotificationFramesAct = 192
08:49:05.495 AudioTrackExtImpl io.element.android.x.nightly D doSchedBoost, set audioapp task boost, pid=29664, tid=31082, enable=0, ret=0
08:49:05.497 AudioTrackExtImpl io.element.android.x.nightly E get fade type, but return null, pid 29664, uid 10357
08:49:05.497 AudioTrack io.element.android.x.nightly D set: pop_optimize mIsPopOptimizeSupport:0, mIsPopOptimizeFlush:0
08:49:05.497 AudioTrack io.element.android.x.nightly D setVolume(6568) left 1.000 right 1.000 , callingPid 29664
08:49:05.498 AAudioStream io.element.android.x.nightly D setState(s#7) from 0 to 2
08:49:05.501 AAudio io.element.android.x.nightly I AAudioStreamBuilder_openStream() returns 0 = AAUDIO_OK for s#7 ----------------
08:49:05.502 AAudio io.element.android.x.nightly D AAudioStream_requestStart(s#7) called --------------
08:49:05.502 AAudioStream io.element.android.x.nightly D setState(s#7) from 2 to 3
08:49:05.502 AudioTrack io.element.android.x.nightly D start(6568): prior state:STATE_STOPPED output 13 stream 0 session 17689
08:49:05.521 AppOps system_server E attributionTag not declared in manifest of io.element.android.x.nightly
08:49:05.522 AppOps system_server E attributionTag not declared in manifest of io.element.android.x.nightly
08:49:05.523 AAudio io.element.android.x.nightly D AAudioStream_requestStart(s#7) returned 0 ---------
08:49:05.537 AAudioStream io.element.android.x.nightly D setState(s#3) from 3 to 4
08:49:05.539 AudioStreamLegacy io.element.android.x.nightly D onAudioDeviceUpdate() devices 2 => 2
08:49:05.543 AAudioStream io.element.android.x.nightly D setState(s#7) from 3 to 4
08:49:05.604 AAudio io.element.android.x.nightly D AAudioStream_requestStop(s#7) called
08:49:05.604 AAudioStream io.element.android.x.nightly D setState(s#7) from 4 to 9
08:49:05.604 AudioTrack io.element.android.x.nightly D [oplusAudioFade]stop mState STATE_ACTIVE, mIsTrackSupportFade 0
08:49:05.604 AudioTrack io.element.android.x.nightly D stop(6568): prior state:STATE_ACTIVE output 13 stream 0 session 17689
08:49:05.604 AudioTrack io.element.android.x.nightly D stop(6568): called with 3648 frames delivered
08:49:05.605 AppOps system_server E attributionTag not declared in manifest of io.element.android.x.nightly
08:49:05.605 AppOps system_server E attributionTag not declared in manifest of io.element.android.x.nightly
08:49:05.605 AudioTrack io.element.android.x.nightly I stop(6568): 0xb400007a8e251400 stop done
08:49:05.606 AAudioStream io.element.android.x.nightly D setState(s#7) from 9 to 10
08:49:06.648 AAudio io.element.android.x.nightly D AAudioStream_requestStop(s#3) called
08:49:06.648 AAudioStream io.element.android.x.nightly D setState(s#3) from 4 to 9
08:49:06.648 AudioRecord io.element.android.x.nightly D stop(6565): mActive:1
08:49:06.648 org.matrix.rust.sdk io.element.android.x.nightly D elementx: [ElementCall] :2 [Lifecycle] LobbyView Component unmounted |
08:49:06.648 AudioRecord io.element.android.x.nightly D call mAudioRecord->stop()
08:49:06.654 org.matrix.rust.sdk io.element.android.x.nightly D elementx: [ElementCall] :2 [InCallView] START CALL VIEW SCOPE |
```

Note these logs were taken directly from the device: rageshakes don't contain info about the internal logs for the OS, missing most of the `AAudio`, `AAudioStream` and `AudioRecord` logs.

Looking for more info about this issue, I found someone who had a similar issue and apparently solved it by manually passing the device id as a constraint when using `MediaDevices.getUserMedia(constraints)`: https://issuetracker.google.com/issues/204914497#comment41. I'm not sure how feasible that is, but it does seem worth a try if we can make it work through EC -> LiveKit's JS SDK.

Also, in the device that mostly works, if you mute the microphone in the lobby before joining the call, most of the time the media volume will be used again: it turns out using `AudioManager.mode = AudioManager.MODE_IN_COMMUNICATION` will only be effective for up to 5-6s if no audio is played (or recorded?) in the meantime, after which it will come back to `AudioManager.mode = AudioManager.MODE_NORMAL` and it will use the media volume again. Some devs mentioned a silent audio could be played in the Android side to prevent this automatic mode change (a media player with an audio stream that's completely empty and loops), this is something we could implement on the Android side.

### Operating system

Android 15, 16

### Browser information

Android WebView 149.0.7827.91 (Chromium)

### URL for webapp

Embedded in Element X Android

### Will you send logs?

Yes

Contributor guide

Open the contributing guide

Research direction

Start by reproducing a call on the Xiaomi Mi 9 or Poco F6 with Android 15 and compare its audio logs with the working OnePlus logs. Investigate how AudioManager.mode and the AAudio or AudioTrack paths affect routing; done means call audio is controlled by the call-volume rocker when starting or joining a call.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, typescript
Domain
audio-video-rtc, mobile-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.