Unrecoverable SocketTimeoutExceptions when playing live streams
@icbaker is already working on this.
Since Oct 12, 2023.
- Dominant language
- Java
- Stars
- 3k
- Forks
- 955
- Avg merge
- 12d 14h
- Merged PRs (30d)
- 2
Description
This probably should be a bug report. But that template is too hard to edit.
All testing are based on media3 1.1.1 and okhttp 4.11.0.
Our live streams are based short DASH or HLS manifests of about two minutes long. So when users perform seeks on our programs, we need to create new stream urls and send them to exoPlayer. That means every time user seeks, we need to create a new MediaItem instance and call exoPlayer.setMediaItem() and exoPlayer.prepare().
Users have been reporting random network connection issues (not real network issues of course) and the only way to recover is to force kill and restart the app. I am also able to reproduce this issue randomly. It turns out exoPlayer randomly throws some SocketTimeoutExceptions if we perform exoPlayer.setMediaItem() and exoPlayer.prepare() repeatedly on the same exoPlayer instance.
media3 had this commit to workaround some issues in okhttp regarding thread interruption: https://github.com/androidx/media/commit/80928e730c53729d147264a910fb327ba88be257. But looks like media3 is still using thread interruptions here and there. For example:
The stack trace is:
Playback error
androidx.media3.exoplayer.ExoPlaybackException: Source error
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:684)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:656)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loopOnce(Looper.java:205)
at android.os.Looper.loop(Looper.java:294)
at android.os.HandlerThread.run(HandlerThread.java:67)
Caused by: androidx.media3.datasource.HttpDataSource$HttpDataSourceException: java.net.SocketTimeoutException: timeout
at androidx.media3.datasource.okhttp.OkHttpDataSource.read(OkHttpDataSource.java:413)
at androidx.media3.datasource.StatsDataSource.read(StatsDataSource.java:94)
at androidx.media3.exoplayer.source.chunk.DataChunk.load(DataChunk.java:98)
at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:415)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at java.lang.Thread.run(Thread.java:1012)
Caused by: java.net.SocketTimeoutException: timeout
at okhttp3.internal.http2.Http2Stream$StreamTimeout.newTimeoutException(Http2Stream.kt:675)
at okhttp3.internal.http2.Http2Stream$StreamTimeout.exitAndThrowIfTimedOut(Http2Stream.kt:684)
at okhttp3.internal.http2.Http2Stream$FramingSource.read(Http2Stream.kt:379)
at okhttp3.internal.connection.Exchange$ResponseBodySource.read(Exchange.kt:281)
at okio.RealBufferedSource$inputStream$1.read(RealBufferedSource.kt:158)
at androidx.media3.datasource.okhttp.OkHttpDataSource.readInternal(OkHttpDataSource.java:579)
at androidx.media3.datasource.okhttp.OkHttpDataSource.read(OkHttpDataSource.java:406)
at androidx.media3.datasource.StatsDataSource.read(StatsDataSource.java:94)
at androidx.media3.exoplayer.source.chunk.DataChunk.load(DataChunk.java:98)
at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:415)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
at java.lang.Thread.run(Thread.java:1012)
I added some logging code in media3 1.1.1 so the line number doesn't match original 1.1.1 tag. The Loader.txt and OkhttpDataSource.txt are the modified source files.
Loader.txt
OkhttpDataSource.txt
The logcats.txt is the corresponding logcat printed when this issue is reproduced.
I also added a break pointer at Thread.interrupt() method with stack trace dump and evaluation of expression at the start of Thread.interrupt method:
"At: " + System.currentTimeMillis() + " Thread " + Thread.currentThread().getId() + " " + Thread.currentThread().getName() + " try to interrupt " + this.getId() + " " + this.getName()
ThreadInterrupts.txt
When these SocketTimeoutExceptions happen, the data probably reaches the device correctly (since the 200 code and correct ResponseBody length). The method OkhttpDataSource.open parses out the 200 response code and correct body length. But when OkhttpDataSource.read is called, the method throws IOExceptions due to thread interruptions. I think when you call exoPlayer.prepare with new MediaItems frequently, there are some thread contentions.
For example, in the logcat file at 15:51:33.318, the call to OkhttpDataSource.open is successful with correct body length 16, but the following OkhttpDataSource.read is blocked and eventually causes a time out after 10 seconds.
What's the suggested way to work around these SocketTimeoutExceptions?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.