jMonkeyEngine / jMonkeyEngine/jmonkeyengine

Problem with multisample postprocessing

Open
#1,246 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
4.3k
Forks
1.2k
Avg merge
4d 7h
Merged PRs (30d)
14

Description

This ticket is the result of the discussion in Discord and its conclusions.

There appears to be an inconsistence within JME postprocessing stack, related to multisampling, that, according to my preliminary findings, encompasses such problems as:

  1. Some of the stock shaders are not built to support multisampling (for example, FXAA), thus when multisampling is enabled, these shaders fail.
  2. The failure looks like replacing the screenbuffer texture with some randrom texture from the loaded ones, thus effectively breaking the rendering pipeline.
  3. There's no warning or error that notifies the user that a particular shader is not compatible with multisampling and cannot be used.
  4. There's no documentation notice about what shaders allow that usage or not.
  5. Unaware contributors make shaders that are incompatible with multisampling. That could have been avoided if the contribution guide included the relevant section for shaders on how to make them multisampling-compatible.
  6. The problem can be silently "eliminated" by preceeding the line of post effects of the affected FilterPostProcessor with one of the shaders that was by chance made compatible with multisampling. That is not a real solution, however, it's currently widely adopted among the community on a level of a belief that a lucky sequence of postprocessing effects will work and the user just needs to find it by randomly changing the order of the filters in the postprocessing stack until it works.

Reproducing the problem:

The simplest reproduction of the problem can be made by creating a FilterPostProcessor within any JME project (for example, TestPBRLighting or a barest project with a box and a sky sphere and some textures), setting the number of samples on application settings to, for example, 4, setting the number of samples on the FilterPostProcessor to the same amount, and assigning two filters to it:

  1. FXAAFilter
  2. ToneMapFilter

which will lead to the viewport screenbuffer to be replaced with the texture of the environment map:

image

while the reverse combination of the filters will seemingly work:

image

The issue is also reproduced and confirmed by Oxplay on his machine.

The statement of this issue implies that preceeding the FilterPostProcessor chain of effects with even a passthrough filter that simply does nothing except for treating the possibility of multisampling correctly does fix the symptom is confirmed.

This project https://github.com/noncom/jme-multisampling-stopgap contains such a proof
of a passthrough filter that can be used at the start of the chain with any number of following filters that don't support multisampling and eliminate the glitch.

Take note that that filter also requires

 #extension GL_ARB_texture_multisample : enable

which in turn requires specifying GLSL150 version requirement in its j3md.

However, such an "approach" is not a solution, since the problem clearly lies within the JME pipeline or FilterPostProcessor pipeline.

As of now I did not yet get into how the pipelines work inside so I don't know why exactly this "approach" fixes the texture glitch. My current guess is that such a filter simply converges the multiple texture samples into one, thus providing a sampler2D as its output, that is later nonchalantly used by the following filters. If this is true, then the following filters are already silently receiving the color data collapsed to a single sample and might not always be desirable? This must be a problem?

I am not sure how this could have been missed until now, but this looks like a huge gap in the solidity of documentation and/or rendreing/postprocessing stacks and requires some attention.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with FilterPostProcessor using FXAAFilter and ToneMapFilter, first in TestPBRLighting or a minimal project with multisampling enabled. Read the FilterPostProcessor and affected filter shaders, including their j3md files; done should explain and correct multisampling handling, then document compatible shaders and contribution guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
computer-graphics
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.