openframeworks / openframeworks/openFrameworks

ofGstVideoPlayer doesn't handle HLS video stream size changes, and sometimes crashes on change.

Open
#5,308 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
10.4k
Forks
2.6k
Avg merge
1d 21h
Merged PRs (30d)
9

Description

I'm currently streaming from this HLS playlist.

http://iphone-streaming.ustream.tv/uhls/17074538/streams/live/iphone/playlist.m3u8

grabber.setPlayer(std::make_shared<ofGstVideoPlayer>());
grabber.load("http://iphone-streaming.ustream.tv/uhls/17074538/streams/live/iphone/playlist.m3u8");
grabber.play();

It's an HLS playlist. Initially it loads the first listed stream (RESOLUTION=854x480) and after a few seconds it (appears) to jump to the higher res stream (RESOLUTION=1280x720). But the pixels / texture do not appear to resize. The result is the 1280x720 video cropped to 854x480.

In some cases, which I have not been able to reproduce reliably (but I believe it has something to do with this resizing behavior), I will get back access exceptions in the buffer_cb when gst is trying to write an aligned pixels buffer to ofPixels.

There are plenty of interesting unhandled messages coming from ofGstUtils::gstHandleMessage() including GST_MESSAGE_STREAM_STATUS, GST_MESSAGE_ELEMENT, GST_MESSAGE_STREAM_START and others.

I'm happy to dig into this, but, wondering where to dig into this one. It seems to me that when a new buffer is processed, we should be able to simply inspect the meta-data and reallocate the player's ofPixels / ofTexture to match, but I'm not sure. Any thoughts are appreciated.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing ofGstVideoPlayer's buffer_cb and the handling around ofPixels and ofTexture when the HLS stream changes resolution. Review ofGstUtils::gstHandleMessage() and the unhandled GStreamer messages mentioned in the report. Done means resolution changes resize the pixel and texture buffers correctly, the video is not cropped, and buffer_cb no longer causes access exceptions.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
audio-video-rtc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.