KhronosGroup / KhronosGroup/WebGL

Multisampled multiview support

Open
#2,912 6 comments 6 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
2.9k
Forks
703
Avg merge
2d 9h
Merged PRs (30d)
4

Description

We need to support multisampled multview rendering, but how to do so is tricky.

Since we are using texture arrays for multiview, there are two options:
* Multisampled framebuffers: [OVR_multiview_multisampled_render_to_texture](https://www.khronos.org/registry/OpenGL/extensions/OVR/OVR_multiview_multisampled_render_to_texture.txt)
* Multisampled texture arrays: [EXT_multiview_texture_multisample](https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_multiview_texture_multisample.txt)

EXT_multiview_texture_multisample has a good brief on the two:
> This is one of two extensions that allow multisampling when using
> OVR_multiview. Each supports one of the two different approaches to
> multisampling in OpenGL and OpenGL ES:
>
> > Core OpenGL and OpenGL ES 3.1+ have explicit support for multisample
> texture types, such as TEXTURE_2D_MULTISAMPLE. Applications can access
> the values of individual samples and can explicitly "resolve" the
> samples of each pixel down to a single color.
>
> > The extension EXT_multisampled_render_to_texture provides support for
> multisampled rendering to non-multisample texture types, such as
> TEXTURE_2D. The individual samples for each pixel are maintained
> internally by the implementation and can not be accessed directly
> by applications. These samples are eventually resolved implicitly to
> a single color for each pixel.
>
> This extension supports the first multisampling style with multiview
> rendering; the OVR_multiview_multisampled_render_to_texture extension
> supports the second style. Note that support for one of these multiview
> extensions does not imply support for the other.

OVR_multiview_multisampled_render_to_texture was first-to-market, and is generally better supported on ES, particularly ES3.0. OVR_multiview_multisampled_render_to_texture is possible to implement on top of EXT_multisampled_render_to_texture, but not, strictly speaking, vice-versa.

The tricky part is the behavior of implicit resolve in OVR_multiview_multisampled_render_to_texture, as described in [EXT_multisampled_render_to_texture](https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_multisampled_render_to_texture.txt):
> When the texture level is used as a source or destination for any
operation, the attachment is flushed, or when the attachment is broken,
the GL implementation may discard the contents of the implicit multisample
buffer. If the contents are discarded, the subsequent operations on the
multisample buffer will behave as if all samples within a pixel have the
value most recently written to the color buffer for that pixel.

> The operations which may cause the contents of the implicit multisample
buffer to be discarded include:
> * Drawing with the texture bound to an active texture unit
> * ReadPixels or CopyTex[Sub]Image* while the texture is
attached to the framebuffer
> * CopyTex[Sub]Image*, Tex[Sub]Image*,
CompressedTex[Sub]Image* with the specified level as
destination
> * GenerateMipmap
> * Flush or Finish while the texture is attached to the
framebuffer
> * BindFramebuffer while the texture is attached to the currently
bound framebuffer.

Can we as the browser tell when this happens or prevent it? No and no:
> 2. Should there be a way for applications to query if the multisample
> buffer has been resolved - and therefore is undefined?
>
> This may be useful if the operations that cause the multisample
> buffer to be resolved is allowed to vary between implementations.
>
> RESOLVED: No, for two reasons: 1) This extension aims to be backwards
> compatible with the IMG_multisampled_render_to_texture extension, which
> did not include such a query, and 2) Given the resolution of issue 3 this
> is not very useful as the application cannot control whether multisample
> information is preserved or not.
>
> 3. Should there be a way for applications to preserve the multisample
> buffer after a resolve?
>
> This would be similar in spirit to the EGL_BUFFER_PRESERVED options in
> EGL 1.4. Applications could - at a performance and memory cost - choose
> to make the multisample buffer _not_ undefined after a resolve.
>
> RESOLVED: No. The purpose of this extension is to support multisampled
> rendering in a lightway manner. Preserving the multisample buffer goes
> against this intent.

Mid-render resolves cause shimmering artifacts between frames that are interrupted in different places. These artifacts can be acceptable for occasional frames, but it does violate portability of results. (It's these same artifacts that make it inviable to implement EXT_multiview_texture_multisample in terms of OVR_multiview_multisampled_render_to_texture)

It'll take some work in Firefox's WebGL implementation to ensure that we aren't incurring more implicit resolves than we have to, and even then, there will be times we can't protect from implicit resolves. (there are other OpenGL contexts rendering, after all!)

Can we use OVR_multiview_multisampled_render_to_texture to support multisampled multiview? Yes, but imperfectly, with important caveats.
Do we have much of a choice? Probably not, unless we're willing to forgo support on a lot of (most GLES) devices.

@RafaelCintron @kenrussell @grorg

Contributor guide

Open the contributing guide

Research direction

Start by comparing the OVR_multiview_multisampled_render_to_texture and EXT_multiview_texture_multisample specifications, especially their resolve behavior. Done means defining and implementing a WebGL approach that supports multisampled multiview with documented portability and implicit-resolve caveats.

Written by the indexing model from the issue text.

Assessment

Domain
computer-graphics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.