androidx / androidx/media

Low-latency RTSP or UDP playback

Open
#1,179 5 comments 3 reactions 1 assignee View on GitHub

@microkatz is already working on this.

Since Mar 14, 2024.

enhancement low priority
Dominant language
Java
Stars
3k
Forks
955
Avg merge
12d 14h
Merged PRs (30d)
2

Description

The environment is a MAVLink controller (based on Android 10) and Drone setup, the video stream arrives on a local UDP port. Using ExoPlayer part of media3 v1.2.1.


    private val videoRenderer = object : MediaCodecVideoRenderer(context,
        MediaCodecSelector { mimeType, requiresSecureDecoder, requiresTunnelingDecoder ->
            MediaCodecSelector.DEFAULT
                .getDecoderInfos(mimeType, requiresSecureDecoder, requiresTunnelingDecoder)
                .filterNot { it.hardwareAccelerated }
                .toMutableList()
        }) {}

...
            val udpUri = "udp://0.0.0.0:${udpPort}"
            val dsf = DataSource.Factory { UdpDataSource() }
            val mediaitem = MediaItem.Builder()
                .setUri(Uri.parse(udpUri))
                .setLiveConfiguration(
                    MediaItem.LiveConfiguration.Builder().setMaxPlaybackSpeed(1.02f).build()
                )
                .build()
            val mediaSource = ProgressiveMediaSource.Factory(dsf).createMediaSource(mediaitem)
            // Force software rendering to ensure frames arrive in standard format.
            val renderersFactory = RenderersFactory { _,_,_,_,_ ->
                arrayOf(videoRenderer)
            }
            // Disable buffering behavior, we want the fastest possible updated stream.
            val loadControl = DefaultLoadControl.Builder()
                            .setBufferDurationsMs(0, 500, 0, 0)
                            .setPrioritizeTimeOverSizeThresholds(true)
                            .build()
                val player = ExoPlayer.Builder(context)
                    .setRenderersFactory(renderersFactory)
                    .setLoadControl(loadControl)
                    .build()
                player.setMediaSource(mediaSource)
                player.setVideoSurface(imageReaderHelper.surface)
                player.prepare()

We are seeing 2-3 seconds delay between what actually happens and what we see rendered by ExoPlayer in the ImageReader.

We do not mind if ExoPlayer drops frames to achieve lower latency.

Could you please help us with this matter? Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.