video-dev / video-dev/hls.js

Graceful handling of corrupted video fragments

Open
#7,064 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature proposal
Dominant language
TypeScript
Stars
16.9k
Forks
2.8k
Avg merge
1d 12h
Merged PRs (30d)
27

Description

Is your feature request related to a problem? Please describe.

In case of corrupted video segment produced by backend and included in HLS playlist there is no public API to recover from this situation (or I didn't find it?).
I've encountered case when backend produced corrupted fragment, which is accepted by hls.js and passed into <video> element to play.
The video data inside chunk corrupted in such a way that it is broken in the middle of a chunk and causes decoders inside browser to reject video data and <video> element fail with MEDIA_ERR_DECODE error.
Just invoking hls.recoverMediaError() does not help because it seems to still feed invalid data into decoder causing it fail again and again.

Describe the solution you'd like

I've found the following way to recover from such errors in context of modified demo app, but it abuses ability to call private api from JS, so there should be a better way to just skip broken fragment and move to next one.

function handleMediaError() {
...
      $('#statusOut').append(', trying to recover media error.');

      const video = hls.media;
      if (video) {
        const time = video.currentTime;
        const fragPlaying = hls.streamController.currentFrag;
        if (fragPlaying && fragPlaying.startPTS <= time && time <= fragPlaying.endPTS) {
          // Reset the broken state of HTMLVideoElement
          // hls.media.load();
          // Effectively skip the rest of the media in currently playing chunk.
          hls.media.currentTime = fragPlaying.endPTS + 0.001;
        } else {
          console.warn(`Inconsistency between hls.media.currentTime=${time} and [fragPlaying.startPTS=${fragPlaying.startPTS}; fragPlaying.endPTS=${fragPlaying.endPTS}], skip 1 seconds of media`);
          hls.media.currentTime = time + 1;
        }
      }

      hls.recoverMediaError();
....
}

I've also played a bit with hls.js itself and implemented similar workaround within recoverMediaError which automatically skips failed fragment when <video> element fails with decode error:

  recoverMediaError() {
    this.logger.log('recoverMediaError');

    const resumeTime = (media: HTMLMediaElement) => {
      const mediaTime = media.currentTime;
      if (media.error?.code == MediaError.MEDIA_ERR_DECODE) {
        let fragPlaying = this.streamController.currentFrag;
        if (fragPlaying && fragPlaying.startPTS && fragPlaying.endPTS && fragPlaying.startPTS <= mediaTime && mediaTime <= fragPlaying.endPTS) {
          // Skip fragment which is broken
          return fragPlaying.endPTS + 0.001;
        } else {
          return media.currentTime + this.config.nudgeOffset;
        }
      } else {
        return media.currentTime
      }
    }

    const media = this._media;
    const time = media ? resumeTime(media) : null;
    this.detachMedia();
    if (media) {
      this.attachMedia(media);
      if (time) {
        media.currentTime = time;
        this.startLoad(time);
      }
    }
  }

does it looks like something which could be accepted to upstream?

Additional context

I've prepared repository with example of such broken chunks (329.ts broken in the middle). It also have associated github pages which could be used to host playlist and chunks so it's possibly to try reproducible broken stream on demo app permalink.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the public recoverMediaError entry point and the streamController.currentFrag state described in the issue. Reproduce the failure using the linked broken-fragment repository and demo playlist, then inspect how recovery handles MEDIA_ERR_DECODE. Done means a documented public recovery path can skip the corrupted fragment without repeatedly feeding it to the decoder.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
audio-video-rtc
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.