RTSP stream with video stops working and freezes on first frame when audio is added

Version

Media3 1.2.0

The issue/bug details

I have set up a local RTSP stream with GStreamer, and are experiencing some problems when trying to stream the audio.
What happens is that the stream wont play in ExoPlayer (it works in VLC using libVLC) when I try to stream the audio, this happens both when there is only audio, and if I play video+audio (only video works fine in ExoPlayer).

I have checked that the audio is AAC (using ffmpeg) in my mp4-file that I stream, but maybe GStreamer does something to it since it's not working in ExoPlayer.
However if I add some extra parameters when setting up the stream (not just using raw video and audio) and adding some "parsing to aac" (in GStreamer python script) the RTSP stream works with both video and audio in ExoPlayer.

Im very new to streaming and might misunderstand what audio-types that is compatible with ExoPlayer and not, but as mentioned the stream which was sent in its simplest form worked in VLC and not in ExoPlayer, which I find odd since its a common formatted stream.

Some logging

Here is some logging from Android Studio, the first one is for the not working stream and the second is for the working stream (Note that the parameter "bitrate" is present only in the working stream):

Stream that doesn't work
tracks [eventTime=0.37, mediaPos=0.00, window=0, period=0
group [
[X] Track:0, id=null, mimeType=video/avc, codecs=avc1.4d401e, res=640x360, color=BT709/Limited range/SDR SMPTE 170M/8/8, supported=YES
]
group [
[X] Track:0, id=null, mimeType=audio/mp4a-latm, codecs=mp4a.40.30, channels=2, sample_rate=44100, supported=YES
]
]

Stream that works
tracks [eventTime=0.47, mediaPos=0.00, window=0, period=0
group [
[X] Track:0, id=null, mimeType=video/avc, codecs=avc1.4d401e, res=640x360, color=BT709/Limited range/SDR SMPTE 170M/8/8, supported=YES
]
group [
[X] Track:0, id=null, mimeType=audio/mp4a-latm, bitrate=128000, codecs=mp4a.40.2, channels=2, sample_rate=44100, supported=YES
]
]

Devices that reproduce the issue

Samsung Galaxy S22 running Android 14
Pixel 2 emulator running Android 11

Devices that do not reproduce the issue

None (so far)

Reproducible in the demo app?

Not tested (couldn't find test for RTSP stream)

Reproduction steps

For me I had to set up a RTSP stream of a video file that had AAC audio (I used a mp4 video file), and that would not be playable in ExoPlayer.
I used GStreamer and/or VLC on Mac to set up the local streams.

Expected result

The video should play with audio (synced to the video)

Actual result

The first frame of the video is shown, and the max duration is inserted into the "duration" element in the ExoPlayer controller, but the current duration is stuck at 0 (not playing).

Media

https://github.com/androidx/media/assets/60511356/67060b6d-0289-4a83-ab31-731506ef3f87