Low-latency RTSP or UDP playback
Open
@microkatz is already working on this.
Since Mar 14, 2024.
enhancement
low priority
- Dominant language
- Java
- Stars
- 3k
- Forks
- 955
- Avg merge
- 12d 14h
- Merged PRs (30d)
- 2
Description
The environment is a MAVLink controller (based on Android 10) and Drone setup, the video stream arrives on a local UDP port. Using ExoPlayer part of media3 v1.2.1.
private val videoRenderer = object : MediaCodecVideoRenderer(context,
MediaCodecSelector { mimeType, requiresSecureDecoder, requiresTunnelingDecoder ->
MediaCodecSelector.DEFAULT
.getDecoderInfos(mimeType, requiresSecureDecoder, requiresTunnelingDecoder)
.filterNot { it.hardwareAccelerated }
.toMutableList()
}) {}
...
val udpUri = "udp://0.0.0.0:${udpPort}"
val dsf = DataSource.Factory { UdpDataSource() }
val mediaitem = MediaItem.Builder()
.setUri(Uri.parse(udpUri))
.setLiveConfiguration(
MediaItem.LiveConfiguration.Builder().setMaxPlaybackSpeed(1.02f).build()
)
.build()
val mediaSource = ProgressiveMediaSource.Factory(dsf).createMediaSource(mediaitem)
// Force software rendering to ensure frames arrive in standard format.
val renderersFactory = RenderersFactory { _,_,_,_,_ ->
arrayOf(videoRenderer)
}
// Disable buffering behavior, we want the fastest possible updated stream.
val loadControl = DefaultLoadControl.Builder()
.setBufferDurationsMs(0, 500, 0, 0)
.setPrioritizeTimeOverSizeThresholds(true)
.build()
val player = ExoPlayer.Builder(context)
.setRenderersFactory(renderersFactory)
.setLoadControl(loadControl)
.build()
player.setMediaSource(mediaSource)
player.setVideoSurface(imageReaderHelper.surface)
player.prepare()
We are seeing 2-3 seconds delay between what actually happens and what we see rendered by ExoPlayer in the ImageReader.
We do not mind if ExoPlayer drops frames to achieve lower latency.
Could you please help us with this matter? Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.