Does H264Reader handle midstream SPS/PPS (from TV broadcast) correctly?

I'm using ExoPlayer 2.18.1, and I noticed an oddity when playing a TV broadcast capture (h264/ac3) related to the way the h264 stream is segmented by H264Reader to be fed to the platform codec.

(I will send a link to the sample stream by email to devs. Note that the sample stream requires `FLAG_ALLOW_NON_IDR_KEYFRAMES`.)

The sample stream has the particularity to have midstream SPS/PPS at regular intervals, isolated between AUD's. That is, one can find the following NAL pattern, at regular intervals (I checked with https://mradionov.github.io/h264-bitstream-viewer/):
- AUD
- SPS (payload size = 28)
- PPS (payload size = 4)
- AUD
- SPS (payload size = 28)
- PPS (payload size = 4)
- SEI (payload size = 5)
- SEI (payload size = 5)
- SEI (payload size = 5)
- non-IDR
- AUD

H264Reader segments the above in 2 chunks, the first one of size 46 bytes, and the second one with a size dependent on the non-IDR slice. I'm testing on a platform that uses ffmpeg-based codecs, and such segmentation leads to an error in the decoder, because I think ffmpeg expects stream to be segmented at frame boundaries, and the first chunk above does not contain any frame.

The platform also contains an ffmpeg-based extractor (which is used by gallery3d player), and when using it, the extractor actually only produces one chunk for the entire above sequence, which matches what ffmpeg codec expects.

So I've experimented with the following patch, which solves the decoding error when using ExoPlayer, without any noticeable adverse effects on other h264 sample streams:
```
diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/H264Reader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/H264Reader.java
index ea5f725ca0..4d5659c5ae 100644
--- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/H264Reader.java
+++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/H264Reader.java
@@ -291,6 +291,7 @@ public final class H264Reader implements ElementaryStreamReader {
private long samplePosition;
private long sampleTimeUs;
private boolean sampleIsKeyframe;
+ private boolean sampleIsFrame;

private int sampleCounter;

@@ -483,16 +484,18 @@ public final class H264Reader implements ElementaryStreamReader {

public boolean endNalUnit(
long position, int offset, boolean hasOutputFormat, boolean randomAccessIndicator) {
- if (nalUnitType == NalUnitUtil.NAL_UNIT_TYPE_AUD
+ if ((nalUnitType == NalUnitUtil.NAL_UNIT_TYPE_AUD
+ && (!hasOutputFormat || !readingSample || sampleIsFrame))
|| (detectAccessUnits && sliceHeader.isFirstVclNalUnitOfPicture(previousSliceHeader))) {
// If the NAL unit ending is the start of a new sample, output the previous one.
- if (hasOutputFormat && readingSample) {
+ if (hasOutputFormat && readingSample && sampleIsFrame) {
int nalUnitLength = (int) (position - nalUnitStartPosition);
outputSample(offset + nalUnitLength);
}
samplePosition = nalUnitStartPosition;
sampleTimeUs = nalUnitTimeUs;
sampleIsKeyframe = false;
+ sampleIsFrame = false;
readingSample = true;
}
boolean treatIFrameAsKeyframe =
@@ -500,6 +503,9 @@ public final class H264Reader implements ElementaryStreamReader {
sampleIsKeyframe |=
nalUnitType == NalUnitUtil.NAL_UNIT_TYPE_IDR
|| (treatIFrameAsKeyframe && nalUnitType == NalUnitUtil.NAL_UNIT_TYPE_NON_IDR);
+ sampleIsFrame |=
+ nalUnitType >= NalUnitUtil.NAL_UNIT_TYPE_NON_IDR
+ && nalUnitType <= NalUnitUtil.NAL_UNIT_TYPE_IDR;
return sampleIsKeyframe;
}

```

I'm reporting this as a question, not as a bug, simply as a FYI about potential issue with H264 segmentation with this particular TV capture. You may well be considering this is a defect in ffmpeg itself (although I doubt the ffmpeg devs would agree).