google / google/ExoPlayer

RTSP - distorted/pixelated for some URIs

Open
#9,167 5 comments 0 reactions 1 assignee Claimed by @claincly View on GitHub
bug needs triage
Dominant language
Java
Stars
21.9k
Forks
6k
PR merge metrics
No merged PRs in 30d

Description

When trying to stream RTSP using Exoplayer (v2.14.1 & v2.14.0), I encountered an issue where the view was live and moving, however the PlayerView or GLSurfaceView were distorted and pixelated:
![image](https://user-images.githubusercontent.com/87072821/124931113-cf239b00-dff9-11eb-8436-6593fdb135e1.png)
It was not a still image or paused. It was moving, but very distorted like the image above.

Tested with:
Sony Xperia XZ3 H9436
Android 10 API 29
v2.14.1 and v2.14.0 of following libraries:
```Gradle
com.google.android.exoplayer:exoplayer-core:2.14.1
com.google.android.exoplayer:exoplayer-ui:2.14.1
com.google.android.exoplayer:exoplayer-rtsp:2.14.1
```

Here is the rough setup:
```Kotlin
simpleExoPlayer = new SimpleExoPlayer.Builder(getContext()).build();
RtspMediaSource mediaSource = new RtspMediaSource.Factory().createMediaSource(MediaItem.fromUri(uri));
simpleExoPlayer.setMediaSource(mediaSource);
simpleExoPlayer.prepare();
simpleExoPlayer.setPlayWhenReady(true);
//simpleExoPlayer.setVideoSurfaceView(exoPlayerSurfaceView);
exoPlayerView.setPlayer(simpleExoPlayer);
```

Here are the logs to support with investigation:

v2.14.1
DIDN'T WORK - live stream was distorted, using both GLSurfaceView and PlayerView (exoplayer-ui)
```Gradle
2021-07-08 14:55:04.195 26936-26936/com.company.app.cv D/LiveView: onresume: RtspLiveViewSource{mode=Video, sourceType=Rtsp, pipMode=-1, uri=rtsp://192.168.1.254/xxxx.mov},
2021-07-08 14:55:04.195 26936-26936/com.company.app.cv W/tbase.cv: Accessing hidden method Lsun/misc/Unsafe;->getObjectVolatile(Ljava/lang/Object;J)Ljava/lang/Object; (greylist, linking, allowed)
2021-07-08 14:55:04.195 26936-26936/com.company.app.cv W/tbase.cv: Accessing hidden method Lsun/misc/Unsafe;->putOrderedLong(Ljava/lang/Object;JJ)V (greylist, linking, allowed)
2021-07-08 14:55:04.199 26936-26936/com.company.app.cv I/ExoPlayerImpl: Init 59df58 [ExoPlayerLib/2.14.1] [H9436, H9436, Sony, 29]
2021-07-08 14:55:04.203 26936-26936/com.company.app.cv W/tbase.cv: Accessing hidden method Lsun/misc/Unsafe;->putOrderedLong(Ljava/lang/Object;JJ)V (greylist, linking, allowed)
2021-07-08 14:55:04.238 26936-26936/com.company.app.cv D/LeakCanary: Watching instance of androidx.loader.app.LoaderManagerImpl$LoaderViewModel (androidx.loader.app.LoaderManagerImpl$LoaderViewModel received ViewModel#onCleared() callback) with key b8c429ef-df83-4d55-b0df-eca600a84839
2021-07-08 14:55:04.238 26936-26936/com.company.app.cv D/LeakCanary: Watching instance of leakcanary.internal.ViewModelClearedWatcher (leakcanary.internal.ViewModelClearedWatcher received ViewModel#onCleared() callback) with key 0408e5e7-d772-4748-8302-8dc9627cbde5
2021-07-08 14:55:04.238 26936-26936/com.company.app.cv D/LeakCanary: Watching instance of com.company.app.cv.app.ProgressDialogFragment_ (com.company.app.cv.app.ProgressDialogFragment_ received Fragment#onDestroy() callback) with key 60b365ee-8e15-4b63-aa79-e4c58d9385ac
2021-07-08 14:55:04.251 26936-27676/com.company.app.cv W/VideoCapabilities: Unsupported mime image/vnd.android.heic
2021-07-08 14:55:04.302 26936-27682/com.company.app.cv I/OMXClient: IOmx service obtained
2021-07-08 14:55:04.389 26936-27681/com.company.app.cv D/SurfaceUtils: connecting to surface 0x7ce6865010, reason connectToSurface
2021-07-08 14:55:04.389 26936-27681/com.company.app.cv I/MediaCodec: [OMX.qcom.video.decoder.avc] setting surface generation to 27582465
2021-07-08 14:55:04.390 26936-27681/com.company.app.cv D/SurfaceUtils: disconnecting from surface 0x7ce6865010, reason connectToSurface(reconnect)
2021-07-08 14:55:04.390 26936-27681/com.company.app.cv D/SurfaceUtils: connecting to surface 0x7ce6865010, reason connectToSurface(reconnect)
2021-07-08 14:55:04.393 26936-27682/com.company.app.cv I/ExtendedACodec: setupVideoDecoder()
2021-07-08 14:55:04.397 26936-27682/com.company.app.cv I/ExtendedACodec: Decoder will be in frame by frame mode
2021-07-08 14:55:04.433 26936-27682/com.company.app.cv D/SurfaceUtils: set up nativeWindow 0x7ce6865010 for 848x480, color 0x7fa30c06, rotation 0, usage 0x20002900
2021-07-08 14:55:04.608 26936-27682/com.company.app.cv D/SurfaceUtils: set up nativeWindow 0x7ce6865010 for 848x480, color 0x7fa30c06, rotation 0, usage 0x20002900
2021-07-08 14:55:06.921 26936-27012/com.company.app.cv D/LeakCanary: Setting up flushing for Thread[ExoPlayer:Playback,5,main]
2021-07-08 14:55:06.922 26936-27012/com.company.app.cv D/LeakCanary: Setting up flushing for Thread[ExoPlayer:RtspMessageChannel:Sender,5,main]
```

so I thought to try the previous version of the library

v2.14.0
DIDN'T WORK - live stream was distorted, using both GLSurfaceView and PlayerView (exoplayer-ui)
```Gradle
2021-07-08 14:58:29.381 28154-28154/com.company.app.cv D/LiveView: onresume: RtspLiveViewSource{mode=Video, sourceType=Rtsp, pipMode=-1, uri=rtsp://192.168.1.254/xxxx.mov},
2021-07-08 14:58:29.383 28154-28154/com.company.app.cv W/tbase.cv: Accessing hidden method Lsun/misc/Unsafe;->getObjectVolatile(Ljava/lang/Object;J)Ljava/lang/Object; (greylist, linking, allowed)
2021-07-08 14:58:29.383 28154-28154/com.company.app.cv W/tbase.cv: Accessing hidden method Lsun/misc/Unsafe;->putOrderedLong(Ljava/lang/Object;JJ)V (greylist, linking, allowed)
2021-07-08 14:58:29.456 28154-28154/com.company.app.cv W/tbase.cv: Accessing hidden method Landroid/media/AudioTrack;->getLatency()I (greylist, reflection, allowed)
2021-07-08 14:58:29.467 28154-28154/com.company.app.cv I/ExoPlayerImpl: Init 9fae284 [ExoPlayerLib/2.14.0] [H9436, H9436, Sony, 29]
2021-07-08 14:58:29.488 28154-28154/com.company.app.cv W/tbase.cv: Accessing hidden method Lsun/misc/Unsafe;->putOrderedObject(Ljava/lang/Object;JLjava/lang/Object;)V (greylist, linking, allowed)
2021-07-08 14:58:29.491 28154-28154/com.company.app.cv W/tbase.cv: Accessing hidden method Lsun/misc/Unsafe;->putOrderedObject(Ljava/lang/Object;JLjava/lang/Object;)V (greylist, linking, allowed)
2021-07-08 14:58:29.491 28154-28154/com.company.app.cv W/tbase.cv: Accessing hidden method Lsun/misc/Unsafe;->putOrderedLong(Ljava/lang/Object;JJ)V (greylist, linking, allowed)
2021-07-08 14:58:29.540 28154-28154/com.company.app.cv D/LeakCanary: Watching instance of androidx.loader.app.LoaderManagerImpl$LoaderViewModel (androidx.loader.app.LoaderManagerImpl$LoaderViewModel received ViewModel#onCleared() callback) with key 6af26218-e086-4fd9-8e68-5ae71550f210
2021-07-08 14:58:29.540 28154-28154/com.company.app.cv D/LeakCanary: Watching instance of leakcanary.internal.ViewModelClearedWatcher (leakcanary.internal.ViewModelClearedWatcher received ViewModel#onCleared() callback) with key 9cfea660-8ef1-4910-abfb-d5c9cce8c102
2021-07-08 14:58:29.540 28154-28154/com.company.app.cv D/LeakCanary: Watching instance of com.company.app.cv.app.ProgressDialogFragment_ (com.company.app.cv.app.ProgressDialogFragment_ received Fragment#onDestroy() callback) with key 4832d494-d651-4ec4-8254-7f5fae536221
2021-07-08 14:58:29.563 28154-28279/com.company.app.cv W/VideoCapabilities: Unsupported mime image/vnd.android.heic
2021-07-08 14:58:29.613 28154-28287/com.company.app.cv I/OMXClient: IOmx service obtained
2021-07-08 14:58:29.689 28154-28286/com.company.app.cv D/SurfaceUtils: connecting to surface 0x7d5a0a1010, reason connectToSurface
2021-07-08 14:58:29.689 28154-28286/com.company.app.cv I/MediaCodec: [OMX.qcom.video.decoder.avc] setting surface generation to 28829697
2021-07-08 14:58:29.689 28154-28286/com.company.app.cv D/SurfaceUtils: disconnecting from surface 0x7d5a0a1010, reason connectToSurface(reconnect)
2021-07-08 14:58:29.689 28154-28286/com.company.app.cv D/SurfaceUtils: connecting to surface 0x7d5a0a1010, reason connectToSurface(reconnect)
2021-07-08 14:58:29.692 28154-28287/com.company.app.cv I/ExtendedACodec: setupVideoDecoder()
2021-07-08 14:58:29.699 28154-28287/com.company.app.cv I/ExtendedACodec: Decoder will be in frame by frame mode
2021-07-08 14:58:29.751 28154-28287/com.company.app.cv D/SurfaceUtils: set up nativeWindow 0x7d5a0a1010 for 848x480, color 0x7fa30c06, rotation 0, usage 0x20002900
2021-07-08 14:58:30.137 28154-28287/com.company.app.cv D/SurfaceUtils: set up nativeWindow 0x7d5a0a1010 for 848x480, color 0x7fa30c06, rotation 0, usage 0x20002900
2021-07-08 14:58:31.839 28154-28211/com.company.app.cv D/LeakCanary: Setting up flushing for Thread[ExoPlayer:FrameReleaseChoreographer,5,main]
2021-07-08 14:58:31.839 28154-28211/com.company.app.cv D/LeakCanary: Setting up flushing for Thread[ExoPlayer:Playback,5,main]
2021-07-08 14:58:31.839 28154-28211/com.company.app.cv D/LeakCanary: Setting up flushing for Thread[ExoPlayer:RtspMessageChannel:Sender,5,main]
```

It did however work for another more modern camera, but it only worked with library v2.14.0 and not v2.14.1, for which I have opened up another issue for:
https://github.com/google/ExoPlayer/issues/9166

I did also try to disable audio, but didn't have any effect, rough code:
```Kotlin
DefaultTrackSelector selector = new DefaultTrackSelector(getContext());
simpleExoPlayer = new SimpleExoPlayer.Builder(getContext()).setTrackSelector(selector).build();
if (simpleExoPlayer.getRendererType(i) == C.TRACK_TYPE_AUDIO)
selector.setParameters(selector.getParameters().buildUpon().setRendererDisabled(indexOfAudioRenderer, true));
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.