react-native-webrtc / react-native-webrtc/react-native-incall-manager

Reenabling tracks causes lag in remote stream on iOS

Open
#226 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
597
Forks
201
Avg merge
2h 42m
Merged PRs (30d)
1

Description

When InCallManager is started in WebRTC call, it somehow creates lag on unmuting microphone on iOS. Without it everything works fine.

Unmuting with native logs 
[javascript] setting muted to false

(webrtc_voice_engine.cc:1486): Setting voice channel options: AudioOptions {}
(webrtc_voice_engine.cc:424): WebRtcVoiceEngine::ApplyOptions: AudioOptions {audio_jitter_buffer_max_packets: 200, audio_jitter_buffer_fast_accelerate: 0, audio_jitter_buffer_min_delay_ms: 0, }
(webrtc_voice_engine.cc:443): Always disable AEC on iOS. Use built-in instead.
(webrtc_voice_engine.cc:453): Always disable AGC on iOS. Use built-in instead.
(audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform
(audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform
(audio_processing_impl.cc:911): AudioProcessing::ApplyConfig: AudioProcessing::Config{ pipeline: { maximum_internal_processing_rate: 48000, multi_channel_render: 0, multi_channel_capture: 0 }, pre_amplifier: { enabled: 0, fixed_gain_factor: 1 },capture_level_adjustment: { enabled: 0, pre_gain_factor: 1, post_gain_factor: 1, analog_mic_gain_emulation: { enabled: 0, initial_level: 255 }}, high_pass_filter: { enabled: 1 }, echo_canceller: { enabled: 0, mobile_mode: 0, enforce_high_pass_filtering: 1 }, noise_suppression: { enabled: 0, level: High }, transient_suppression: { enabled: 0 }, gain_controller1: { enabled: 0, mode: FixedDigital, target_level_dbfs: 3, compression_gain_db: 9, enable_limiter: 1, analog_gain_controller { enabled: 1, startup_min_volume: 0, clipped_level_min: 70, enable_digital_adaptive: 1, clipped_level_step: 15, clipped_ratio_threshold: 0.1, clipped_wait_frames: 300, clipping_predictor:  { enabled: 0, mode: 0, window_length: 5, reference_window_length: 5, reference_window_delay: 5, clipping_threshold: -1, crest_factor_margin: 3, use_predicted_step: 1 }}}, gain_controller2: { enabled: 0, fixed_digital: { gain_db: 0 }, adaptive_digital: { enabled: 0, headroom_db: 6, max_gain_db: 30, initial_gain_db: 8, max_gain_change_db_per_second: 3, max_output_noise_level_dbfs: -50 }, input_volume_control : { enabled 0}}
(webrtc_voice_engine.cc:1500): Set voice channel options. Current options: AudioOptions {audio_jitter_buffer_max_packets: 200, audio_jitter_buffer_fast_accelerate: 0, audio_jitter_buffer_min_delay_ms: 0, }

Start Monitoring

(RTCLogging.mm:33): (audio_device_ios.mm:444 OnGetPlayoutData): Possible playout audio glitch detected.
  Time since last OnGetPlayoutData was 123 ms.

(RTCLogging.mm:33): (audio_device_ios.mm:453 OnGetPlayoutData): Glitch warning is ignored. Probably caused by device switch.
(RTCLogging.mm:33): (RTCAudioSession.mm:855 -[RTCAudioSession observeValueForKeyPath:ofObject:change:context:]): OutputVolumeDidChange to 0.500000
(RTCLogging.mm:33): (RTCAudioSession.mm:531 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed:
(RTCLogging.mm:33): (RTCAudioSession.mm:544 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: CategoryChange to :AVAudioSessionCategoryMultiRoute
(RTCLogging.mm:33): (RTCAudioSession.mm:563 -[RTCAudioSession handleRouteChangeNotification:]): Previous route: <AVAudioSessionRouteDescription: 0x28165b8d0, 
inputs = (
    "<AVAudioSessionPortDescription: 0x28165a5d0, type = MicrophoneBuiltIn; name = iPad Microphone; UID = Built-In Microphone; selectedDataSource = Front>"
); 
outputs = (
    "<AVAudioSessionPortDescription: 0x28165ab30, type = Speaker; name = Speaker; UID = Speaker; selectedDataSource = (null)>"
)>
Current route:<AVAudioSessionRouteDescription: 0x28164d290, 
inputs = (
    "<AVAudioSessionPortDescription: 0x28164d3c0, type = MicrophoneBuiltIn; name = iPad Microphone; UID = Built-In Microphone; selectedDataSource = Top>"
); 
outputs = (
    "<AVAudioSessionPortDescription: 0x28164cd80, type = Speaker; name = Speaker; UID = Speaker; selectedDataSource = (null)>"
)>

RNInCallManager.AudioRouteChange.Reason: CategoryChange. category=AVAudioSessionCategoryMultiRoute mode=AVAudioSessionModeDefault
RNInCallManager.updateAudioRoute(): [Enter] forceSpeakerOn flag=0 media=video category=AVAudioSessionCategoryMultiRoute mode=AVAudioSessionModeDefault
(RTCLogging.mm:33): (audio_device_ios.mm:630 HandleOutputVolumeChange): Output volume change detected.
RNInCallManager.updateAudioRoute(): audioSession.overrideOutputAudioPort(.None) success
(RTCLogging.mm:33): (audio_device_ios.mm:444 OnGetPlayoutData): Possible playout audio glitch detected.
  Time since last OnGetPlayoutData was 147 ms.

(RTCLogging.mm:33): (audio_device_ios.mm:453 OnGetPlayoutData): Glitch warning is ignored. Probably caused by device switch.
RNInCallManager.updateAudioRoute: audioSession.setCategory: AVAudioSessionCategoryPlayAndRecord, withOptions: 0 success
RNInCallManager.updateAudioRoute() audio category has changed to AVAudioSessionCategoryPlayAndRecord
(RTCLogging.mm:33): (audio_device_ios.mm:444 OnGetPlayoutData): Possible playout audio glitch detected.
  Time since last OnGetPlayoutData was 143 ms.

(RTCLogging.mm:33): (audio_device_ios.mm:453 OnGetPlayoutData): Glitch warning is ignored. Probably caused by device switch.
RNInCallManager.updateAudioRoute: audioSession.setMode(AVAudioSessionModeVideoChat) success
RNInCallManager.updateAudioRoute() audio mode has changed to AVAudioSessionModeVideoChat
RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: End
RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: Unknow Value
(RTCLogging.mm:33): (RTCAudioSession.mm:855 -[RTCAudioSession observeValueForKeyPath:ofObject:change:context:]): OutputVolumeDidChange to 0.750000
(RTCLogging.mm:33): (RTCAudioSession.mm:531 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed:
(RTCLogging.mm:33): (RTCAudioSession.mm:544 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: CategoryChange to :AVAudioSessionCategoryPlayAndRecord
(RTCLogging.mm:33): (RTCAudioSession.mm:563 -[RTCAudioSession handleRouteChangeNotification:]): Previous route: <AVAudioSessionRouteDescription: 0x28164d9a0, 
inputs = (
    "<AVAudioSessionPortDescription: 0x28164dc10, type = MicrophoneBuiltIn; name = iPad Microphone; UID = Built-In Microphone; selectedDataSource = Top>"
); 
outputs = (
    "<AVAudioSessionPortDescription: 0x28164dd90, type = Speaker; name = Speaker; UID = Speaker; selectedDataSource = (null)>"
)>
Current route:<AVAudioSessionRouteDescription: 0x28165b6a0, 
inputs = (
    "<AVAudioSessionPortDescription: 0x281658480, type = MicrophoneBuiltIn; name = iPad Microphone; UID = Built-In Microphone; selectedDataSource = Front>"
); 
outputs = (
    "<AVAudioSessionPortDescription: 0x281659950, type = Speaker; name = Speaker; UID = Speaker; selectedDataSource = (null)>"
)>
RNInCallManager.AudioRouteChange.Reason: CategoryChange. category=AVAudioSessionCategoryPlayAndRecord mode=AVAudioSessionModeVideoChat
RNInCallManager.updateAudioRoute(): [Enter] forceSpeakerOn flag=0 media=video category=AVAudioSessionCategoryPlayAndRecord mode=AVAudioSessionModeVideoChat
RNInCallManager.updateAudioRoute(): audioSession.overrideOutputAudioPort(.None) success
RNInCallManager.updateAudioRoute() did NOT change audio category
RNInCallManager.updateAudioRoute() did NOT change audio mode
RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: End
RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: Unknow Value
(RTCLogging.mm:33): (RTCAudioSession.mm:855 -[RTCAudioSession observeValueForKeyPath:ofObject:change:context:]): OutputVolumeDidChange to 0.900000
(RTCLogging.mm:33): (RTCAudioSession.mm:531 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed:
(RTCLogging.mm:33): (RTCAudioSession.mm:544 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: CategoryChange to :AVAudioSessionCategoryPlayAndRecord
(RTCLogging.mm:33): (RTCAudioSession.mm:563 -[RTCAudioSession handleRouteChangeNotification:]): Previous route: <AVAudioSessionRouteDescription: 0x281649780, 
inputs = (
    "<AVAudioSessionPortDescription: 0x28164ba30, type = MicrophoneBuiltIn; name = iPad Microphone; UID = Built-In Microphone; selectedDataSource = Top>"
); 
outputs = (
    "<AVAudioSessionPortDescription: 0x28164aef0, type = Speaker; name = Speaker; UID = Speaker; selectedDataSource = (null)>"
)>
Current route:<AVAudioSessionRouteDescription: 0x28165a500, 
inputs = (
    "<AVAudioSessionPortDescription: 0x281659ed0, type = MicrophoneBuiltIn; name = iPad Microphone; UID = Built-In Microphone; selectedDataSource = Front>"
); 
outputs = (
    "<AVAudioSessionPortDescription: 0x28165a510, type = Speaker; name = Speaker; UID = Speaker; selectedDataSource = (null)>"
)>
RNInCallManager.AudioRouteChange.Reason: CategoryChange. category=AVAudioSessionCategoryPlayAndRecord mode=AVAudioSessionModeVideoChat
RNInCallManager.updateAudioRoute(): [Enter] forceSpeakerOn flag=0 media=video category=AVAudioSessionCategoryPlayAndRecord mode=AVAudioSessionModeVideoChat
RNInCallManager.updateAudioRoute(): audioSession.overrideOutputAudioPort(.None) success
RNInCallManager.updateAudioRoute() did NOT change audio category
RNInCallManager.updateAudioRoute() did NOT change audio mode
RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: End
RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: Unknow Value
(RTCLogging.mm:33): (audio_device_ios.mm:509 HandleValidRouteChange): RTC_OBJC_TYPE(RTCAudioSession): {
  category: AVAudioSessionCategoryMultiRoute
  categoryOptions: 0
  mode: AVAudioSessionModeVideoChat
  isActive: 1
  sampleRate: 48000.00
  IOBufferDuration: 0.021333
  outputNumberOfChannels: 2
  inputNumberOfChannels: 2
  outputLatency: 0.001458
  inputLatency: 0.000000
  outputVolume: 0.900000
}
(RTCLogging.mm:33): (audio_device_ios.mm:520 HandleSampleRateChange): Handling sample rate change.
(RTCLogging.mm:33): (audio_device_ios.mm:549 HandleSampleRateChange): Handling playout sample rate change:
  Session sample rate: 48000.000000 frames_per_buffer: 1024
  ADM sample rate: 48000.000000 frames_per_buffer: 1024
(RTCLogging.mm:33): (audio_device_ios.mm:554 HandleSampleRateChange): Ignoring sample rate change since audio parameters are intact.
(RTCLogging.mm:33): (audio_device_ios.mm:630 HandleOutputVolumeChange): Output volume change detected.
(connection.cc:1441): Conn[1f5cd400:0:Net[en0:192.168.0.x/24:Wifi:id=1]:YlHZRgcg:1:0:local:udp:192.168.0.x:64990->sZf368Fw:1:2122260223:local:udp:169.254.163.x:57879|C--I|-|0|0|9115038255631187454|-]: Sent STUN BINDING request, id=5752563344375a6f63614f33, use_candidate=1, nomination=0
(RTCLogging.mm:33): (audio_device_ios.mm:509 HandleValidRouteChange): RTC_OBJC_TYPE(RTCAudioSession): {
  category: AVAudioSessionCategoryPlayAndRecord
  categoryOptions: 0
  mode: AVAudioSessionModeVideoChat
  isActive: 1
  sampleRate: 48000.00
  IOBufferDuration: 0.021333
  outputNumberOfChannels: 2
  inputNumberOfChannels: 2
  outputLatency: 0.001458
  inputLatency: 0.000000
  outputVolume: 0.900000
}
(RTCLogging.mm:33): (audio_device_ios.mm:520 HandleSampleRateChange): Handling sample rate change.
(RTCLogging.mm:33): (audio_device_ios.mm:549 HandleSampleRateChange): Handling playout sample rate change:
  Session sample rate: 48000.000000 frames_per_buffer: 1024
  ADM sample rate: 48000.000000 frames_per_buffer: 1024
(RTCLogging.mm:33): (audio_device_ios.mm:554 HandleSampleRateChange): Ignoring sample rate change since audio parameters are intact.
(RTCLogging.mm:33): (audio_device_ios.mm:630 HandleOutputVolumeChange): Output volume change detected.
(RTCLogging.mm:33): (audio_device_ios.mm:509 HandleValidRouteChange): RTC_OBJC_TYPE(RTCAudioSession): {
  category: AVAudioSessionCategoryPlayAndRecord
  categoryOptions: 0
  mode: AVAudioSessionModeVideoChat
  isActive: 1
  sampleRate: 48000.00
  IOBufferDuration: 0.021333
  outputNumberOfChannels: 2
  inputNumberOfChannels: 2
  outputLatency: 0.001458
  inputLatency: 0.000000
  outputVolume: 0.900000
}
(RTCLogging.mm:33): (audio_device_ios.mm:520 HandleSampleRateChange): Handling sample rate change.
(RTCLogging.mm:33): (audio_device_ios.mm:549 HandleSampleRateChange): Handling playout sample rate change:
  Session sample rate: 48000.000000 frames_per_buffer: 1024
  ADM sample rate: 48000.000000 frames_per_buffer: 1024
(RTCLogging.mm:33): (audio_device_ios.mm:554 HandleSampleRateChange): Ignoring sample rate change since audio parameters are intact.

Came from:
https://github.com/react-native-webrtc/react-native-webrtc/issues/1424

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the lag on iOS with InCallManager started, then compare the behavior with InCallManager disabled. Use the supplied WebRTC and audio-session logs to inspect microphone unmuting and audio-route changes; done means unmuting no longer causes lag in the remote stream.

Written by the indexing model from the issue text.

Assessment

Tech stack
ios, react-native
Domain
audio-video-rtc, mobile-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.