KhronosGroup / KhronosGroup/WebGL

Specify and test how texture upload should work with video elements

Open
#3,309 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
2.9k
Forks
703
Avg merge
2d 9h
Merged PRs (30d)
4

Description

Specify and test how texture upload should work with video elements.

https://www.khronos.org/registry/webgl/specs/latest/1.0/#6.9

The obvious expectation would be that the texture would be the size of HTMLVideoElement.videoWidth/videoHeight.

However, texture-corner-case-videos.html does not catch this expectation.

In fact, the test was modified to pass the inverse of this, e.g. behavior where the texture might be smaller.

Also, texture-corner-case-videos seems to contain hidden errors for the texSubImage2D case. The logic is that the texture is created with HTMLVideoElement.videoWidth/videoHeight but then uploaded with texSubImage2D. The upload might be smaller than videoWidth, videoHeight, but this is not caught by the test.

So the ideal intention in #2464 was to let the browser upload video data unscaled 1:1. However, this is problematic as:
1) it is not specified how it works
2) tests about it have bugs
3) it is not possible to use it after it is specified, as the needed property (video data size) is not available in HTMLVideoElement. E.g. you can upload unscaled, but using texSubImage2D to update is impossible as we have no way of knowing what parameters pass as x,y,w,h.

Here's a small tester to print rudimentary cases of how the current browsers work:
[webgl-video-element-texture-size-tester.zip](https://github.com/KhronosGroup/WebGL/files/6953129/webgl-video-element-texture-size-tester.zip)

WebCodecs expose the needed data, but that behavior is not specified in WebGL either.

Contributor guide

Open the contributing guide

Research direction

Read the WebGL specification section 6.9 and inspect texture-corner-case-videos.html, including its texSubImage2D cases. Use the linked tester to compare current behavior. Done means the video texture behavior is specified and the tests correctly catch both texture sizing and subimage upload cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
html
Domain
computer-graphics, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.