openframeworks / openframeworks/openFrameworks

ofBaseVideDraws with internal ofTextures of type OF_PIXELS_YUY2 aren't drawn correctly at custom sizes.

Open
#4,990 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
10.4k
Forks
2.6k
Avg merge
1d 21h
Merged PRs (30d)
9

Description

It seems that the programmable renderer is having some problems rendering YUY2 video textures when asked to do it at widths and heights other than the native video width / height.

For example:

        g = std::make_shared<ofVideoGrabber>();

        // Use the ofxPS3EyeGrabber.
        grabber->setGrabber(std::make_shared<ofxPS3EyeGrabber>());

        // The native pixel format for the ofxPS3EyeGrabber is OF_PIXELS_YUY2
        // (aka YUV422).  When used this way, no additional pixel copies are made
        // or colorspace conversions are performed.
        //
        // The programmable renderer is able to directly render YUV422 pixels.
        // so be sure to that the OpenGL version is > 3.2, otherwise you'll
        // get a blank screen.
        grabber->setPixelFormat(OF_PIXELS_NATIVE); // This means the internal pixel type is OF_PIXELS_YUY2.
        grabber->setup(camWidth, camHeight);
    }

Then we draw

        float w = g->getWidth();
        float h = g->getHeight();

        g->draw(0, 0, w, h);
        g->draw(w, 0, w / 2, h / 2);
        g->draw(0, h, w / 1.5, h / 1.5);

This produces the following output:

screenshot_3_13_16__6_36_pm

So it looks like the internal shaders are correctly drawing the YUY2 planes when the passed video / texture size is the one that is set up in the video player ... but when the size is manually changed (e.g. via draw(), then something gets messed up when drawing these YUY2 textures.

I believe it is happening because when binding the ofBaseVideoDraws, the shader uniforms aren't made aware of the requested non-native video size, leaving the shader to mis-align YUY2 planes when executing the fragment shader.

My investigation is paused here for the moment because I think @arturoc probably has the best angle on this problem :)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the ofBaseVideoDraws example using an OF_PIXELS_YUY2 video texture and the three custom-size draw() calls shown. Inspect the programmable renderer's YUY2 shader path and its handling of video or texture dimensions; done means YUY2 textures render correctly at both native and manually requested sizes.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.