KhronosGroup / KhronosGroup/glTF

Mesh Instancing + LOD + Portals extension proposal

Open
#1,660 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
7.8k
Forks
1.2k
Avg merge
17h 26m
Merged PRs (30d)
5

Description

This is an extension proposal I've been pondering about after my work with [SharpGLTF](https://github.com/vpenades/SharpGLTF), that aims to solve a number of issues in one shot:

- Simplify scene rendering evaluation: makes glTF more _JPEG of 3D_
- Clean LOD support, no weird childs of childs, and much more.
- Multiple meshes per node.

The extension would depend primarily in two objects:

**MeshInstance** would be included in the **Scene** Extensions as a collection.

It essentially defines a Mesh Draw Call:

```c#
MeshInstance
{
int? VisibilityRule; // should we render it?
int Mesh; // what to render.
int? Node; // where to render it (If no Skin).
int? Skin; // how to deform it. (If Skin)
}
```

So, a client that wants to render a scene, instead of traversing the node
tree graph, it would simply loop over the **MeshInstances** of the scene.

Now the interesting part is that a **MeshInstance** references a _VisibilityRule_
object, which are stored in the glTF model extensions.

**VisibilityRule** must be evaluated at runtime to a _Visible | Invisible_ state. If
a **MeshInstance** references a **VisibilityRule** that evaluates to _Invisible_, then that
mesh should not be rendered.

Initially I thought that the rule could simply be the LOD distance or something like
that, but then I noticed that I could be extended to incorporate many other visibility
and occlusion mechanims, other than LOD, so I came with something like that:

```c#
VisibilityRule
{
LODRule LOD;
PortalRule Portal;
... Any other visibility rule we can imagine; Planes? BSP trees?
}

```

Now I am not sure if this is the right way of structuring a VisibilityRule, my
guess is that it is certainly possible to have multiple concurrent rules, for
example, a given rule would only evaluate to visible if it's close enough by a
LOD level, and its portal is visible.

But I believe this mechanism would help into bringing rendering large scenarios
into glTF in a way that is simple to understand and implement.

#### How a client would render the scene?

```c#
for each Node in SCENE.Nodes
{
Node.EvaluateWorldTransform();
}

for each VRule in SCENE.MeshInstances.VisibilityRules
{
VRule.EvaluateVisibility(Projection, View, World);
}

for each MInstance in SCENE.MeshInstances
{
if MInstance.VisibilityRule is Visible
{
if MInstance has Skin RenderSkinnedMesh(MInstance, Skin);
else RenderStaticMesh(MInstance, Node);
}
}
```

The cool thing is that this common implementation would work for normal scenes, scenes with LODs, scenes with Portals, etc.

#### Bonus tracks

- Since the `MeshInstance` List of a scene is basically a collection of drawing calls for a given scene, it is easier to loop over the list, than traverse a Node Tree Graph.

- Also, it is easier to identify which `Mesh` is being rendered multiple times, so it can become a good candidate for GPU instancing.

- Multiple `MeshInstances` can point to the same `Node`, allowing to reference multiple Meshes per Node. Currently it is limited to 1 Mesh per Node.

- It gives more control over how the scene is rendered, simply reordering the items in the MeshInstance collection we can declare the rendering order, irrespectively of how meshes are instantiated in the node tree graph.

- This extension can be backwards compatible as long as the original node tree graph also references the appropiate meshes and skins, but it could be possible to define a node tree graph without mesh references, so the node tree graph would only serve as a mechanism to provide world transforms to the MeshInstances.

- Materials could also take advantage by having an extension pointing to a VisibilityRule and a fallback Material, so when rendering a mesh, if the visibility rule of a given material evaluates to invisible, the fallback material would be used instead. This would bring LODs to materials too.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the glTF scene-extension model and the proposed MeshInstance and VisibilityRule objects. Clarify how concurrent visibility rules, nodes, skins, meshes, LOD, and portals should interact, then define the extension specification and rendering behavior that resolves these open design questions.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
computer-graphics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.