mixed clear and fairplay playback fails

We had an issue on shaka where mixed clear and fairplay content wasn't working in MSE mode but native playback was fine. When trying out hls.js as well, I noticed that it also failed.
There is a related issue for more general drm/clear transition: https://github.com/video-dev/hls.js/issues/4230

Ultimately, @tykus160 was able to figure out that in addition to prepending an enca box to each mdat in the clear segments, the tfhd box needed to be updated to have the sample description index point at the right box. Additionally, the sinf needed to be updated to include the key id from the manifest for the clear segments as well. This should force safari to always be in the DRM realm for playback and not try and switch between the clear and drm realms on the discontinuity. The PR that fixed it on the shaka side.

There is a slack thread with discussion related to this on https://video-dev.slack.com/archives/C4FC8GGEP/p1742940871928379

What version of Hls.js are you using?

latest

What browser (including version) are you using?

Safari 18

What OS (including version) are you using?

macos

Test stream

https://friendly-creponne-991405.netlify.app/fairplay-mixed.m3u8

Configuration

{
  "debug": true,
  "enableWorker": true,
  "lowLatencyMode": true,
  "backBufferLength": 90,
  "drmSystems": {
    "com.apple.fps": {
      "licenseUrl": "https://fps.ezdrm.com/api/licenses/b99ed9e5-c641-49d1-bfa8-43692b686ddb",
      "serverCertificateUrl": "https://fps.ezdrm.com/demo/video/eleisure.cer"
    }
  },
  "emeEnabled": true
}

Additional player setup steps

Using the public fairplay source and another public hls source, I spliced the two together, using a disco tag, for test content.
https://friendly-creponne-991405.netlify.app/mixed-fairplay.m3u8 - encrypted followed by clear
https://friendly-creponne-991405.netlify.app/fairplay-mixed.m3u8 - clear followed by encrypted

https://fps.ezdrm.com/api/licenses/b99ed9e5-c641-49d1-bfa8-43692b686ddb - license server url

https://fps.ezdrm.com/demo/video/eleisure.cer - certificate server url

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Play video through the encryption transition ~6:12min on the fairplay-mixed stream, and ~2min on the mixed-fairplay stream

Expected behaviour

Smooth playback transition.

What actually happened?

Content playback failure. It does work in some cases if seeking into the encrypted section.

Console output

[Log] [log] > – "[transmuxer.ts]: Flushed audio sn: 94 of track 0" (hls.js, line 16706)
[Log] [log] > – "[audio-stream-controller]:" – "PARSING->PARSED" (hls.js, line 10483)
[Log] [log] > – "[audio-stream-controller]:" – "Parsed audio sn: 94 of track 0 (frag:[372.333-378.371])" (hls.js, line 10390)
[Error] [error] > – "[buffer-controller]:" – "Error: audio SourceBuffer error. MediaSource readyState: closed" – Event {isTrusted: true, type: "error", target: ManagedSourceBuffer, …}
Event {isTrusted: true, type: "error", target: ManagedSourceBuffer, currentTarget: ManagedSourceBuffer, eventPhase: 2, …}Event
	onSBUpdateError (hls.js:19817)
[Warning] Error event: – {type: "mediaError", details: "bufferAppendingError", sourceBufferName: "audio", …} (hls-demo.js, line 24810)
{type: "mediaError", details: "bufferAppendingError", sourceBufferName: "audio", error: Error: audio SourceBuffer error. MediaSource readyState: closed, fatal: false, …}Object
[Warning] [warn] > – "[buffer-controller]:" – "Failed 1/3 times to append segment in \"audio\" sourceBuffer" (hls.js, line 19309)
[Warning] [warn] > – "[content-steering]:" – "Could not resolve bufferAppendError (\"audio SourceBuffer error. MediaSource readyState: closed\") with content-steering for Pathway: …" (hls.js, line 21762)
"Could not resolve bufferAppendError (\"audio SourceBuffer error. MediaSource readyState: closed\") with content-steering for Pathway: . levels: 1 priorities: [\".\"] penalized: {\".\":153229}"
[Log] [log] > – "[audio-stream-controller]:" – "Reset loading state" (hls.js, line 10297)
[Log] [log] > – "[audio-stream-controller]:" – "PARSED->IDLE" (hls.js, line 10483)
[Log] [log] > – "stopLoad" (hls.js, line 35378)
[Log] [log] > – "[stream-controller]:" – "FRAG_LOADING->STOPPED" (hls.js, line 10483)
[Log] [log] > – "[audio-stream-controller]:" – "IDLE->STOPPED" (hls.js, line 10483)
[Log] [log] > – "[subtitle-stream-controller]:" – "IDLE->STOPPED" (hls.js, line 10483)
[Warning] Error event: – {type: "mediaError", parent: "audio", details: "bufferAppendError", …} (hls-demo.js, line 24810)
{type: "mediaError", parent: "audio", details: "bufferAppendError", sourceBufferName: "audio", frag: Fragment, …}Object
[Error] Fatal error : bufferAppendError
	(anonymous function) (hls-demo.js:24885)
	emit (hls.js:401)
	onError (hls.js:19314)
	onSBUpdateError (hls.js:19830)
[Warning] [warn] > – "[stream-controller]:" – "Fragment 94 of level 0 was aborted" (hls.js, line 10171)
[Log] [log] > – "[stream-controller]:" – "STOPPED->IDLE" (hls.js, line 10483)
[Log] [log] > – "[buffer-controller]:" – "Media source ended" (hls.js, line 18631)
[Log] [log] > – "[buffer-controller]:" – "Media source closed" (hls.js, line 18628)
[Error] The video playback was aborted due to a corruption problem or because the video used features your browser did not support - Media failed to decode
	handleVideoEvent (hls-demo.js:25036)
[Log] [log] > – "Debug logs enabled for \"main\" in hls.js version 1.6.0" (1d093498-87f0-4f32-9a59-387d06054854, line 1076)
[Log] [log] > – "Debug logs enabled for \"audio\" in hls.js version 1.6.0" (1d093498-87f0-4f32-9a59-387d06054854, line 1076)

Start with the transmuxer and buffer-controller handling of discontinuities, using the supplied fairplay-mixed.m3u8 and mixed-fairplay.m3u8 streams in Safari 18 to reproduce the failure. Compare the clear-segment fragment metadata with the related Shaka fix and verify that playback crosses both encryption transitions without a SourceBuffer or decode error.