EricssonResearch / EricssonResearch/openwebrtc
FPS parameter is ignored in Android
- Dominant language
- C
- Stars
- 1.8k
- Forks
- 531
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I am using openwebrtc for android and I am faced with a strange issue where the stream is sometimes dropped without any error in the log (this happens in my code as well as in the original demo code).
Google suggests that when there is too much data transmitted this can happens and that the solution is to drop the fps.
Thankfully there is a variable for that in the library:
` PeerConnectionParameters params = new PeerConnectionParameters(
true, false, displaySize.x, displaySize.y, fps, 1, VIDEO_CODEC_VP9, true, 1, AUDIO_CODEC_OPUS, true);
webRtcClient = new WebRtcClient((WebRtcClient.RtcListener) mWebRtcContext, mSocketAddress, params, VideoRendererGui.getEGLContext()); `
PeerConnectionParameters takes a fps value. Unfortunately this value seems to be ignored by webrtc - even setting it to 1 does not change anything. Indeed, commenting out the entire parameter setting in the demo code of WebRtcClient does not change anything:
`private void setCamera() {
localMS = factory.createLocalMediaStream("ARDAMS");
if (pcParams.videoCallEnabled) {
MediaConstraints videoConstraints = new MediaConstraints();
//comment out to see what happens
//videoConstraints.mandatory.add(new MediaConstraints.KeyValuePair("maxHeight", Integer.toString((pcParams.videoHeight))));
//videoConstraints.mandatory.add(new MediaConstraints.KeyValuePair("maxWidth", Integer.toString((pcParams.videoWidth))));
//videoConstraints.mandatory.add(new MediaConstraints.KeyValuePair("maxFrameRate", Integer.toString(pcParams.videoFps)));
//videoConstraints.mandatory.add(new MediaConstraints.KeyValuePair("minFrameRate", Integer.toString(pcParams.videoFps)));
VideoCapturer videoCapturer = getVideoCapturer();
videoSource = factory.createVideoSource(videoCapturer, videoConstraints);
localMS.addTrack(factory.createVideoTrack("ARDAMSv0", videoSource));
}
//AudioSource audioSource = factory.createAudioSource(new MediaConstraints());
//localMS.addTrack(factory.createAudioTrack("ARDAMSa0", audioSource));
mListener.onLocalStream(localMS);
}`
Having found that the settings are ignored, how can I change the fps? I tried to follow the trail to see where the video settings are decided but after going through several wrapper classes I was unable to find out. My only hint is this in the log:
`V/MediaCodecVideoEncoder: setRates: 1992 kbps. Fps: 30`
Hundreds of lines like this one, but searching for MediaCodecVideoEncoder in the path did not yield anything, so I suppose it is somewhere in the binaries?
Contributor guide
Research direction
Start by tracing PeerConnectionParameters.videoFps through WebRtcClient.setCamera and the MediaConstraints passed to createVideoSource. Compare that path with the MediaCodecVideoEncoder setRates log, then determine where the requested frame rate is lost or replaced. Done means a configured FPS value changes the encoder behavior and the Android demo no longer reports a fixed 30 FPS.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, java
- Domain
- audio-video-rtc, mobile-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100