KhronosGroup / KhronosGroup/glTF-Sample-Assets

Performance test models

Open
#105 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.1k
Forks
107
Avg merge
9h 39m
Merged PRs (30d)
4

Description

This is related to other issues, but may warrant its own discussion.

- It is related to https://github.com/KhronosGroup/glTF-Sample-Assets/issues/56 , but this focussed on hardware- and API limitations
- It is related to https://github.com/KhronosGroup/glTF/issues/1699 , but this seemed to bring up the question on the level of the specification, without the clear goal of _having_ dedicated test models

It is also related to a recent discussion in https://github.com/KhronosGroup/glTF-Sample-Assets/pull/100 , which aims at having a model that challenges the runtime with many nodes. And I think that it is **good** to have such a model. But I think that it would be even better to not have a _single_ model, but to have models that more or less systematically cover different dimensions along which the performance can be tested.

And there are _many_ dimensions. Paraphrasing from a comment in the linked PR, there could be test models with ...

- 1000, 5000, 50000, and 100000 nodes
- 10000 (or more) nodes that refer to only 10, 100, or 1000 different meshes
- meshes that have 50, 100, 5000, 10000, 65635 vertices
- meshes that use 1, 10, or 1000 different textures
- textures of size 32x32 ... 4096x4096

All these can be combined, yielding a many-dimensional space with billions of _potential_ test models.

The question then is:

**What should be tested?**

I think that people with experience in engine/viewer development might have some ideas what the pain points (i.e. the _interesting_ points) could be. And with a clearer idea about these interesting points, one could set up a collection of test models. (Not necessarily as actual assets - preferably, with a mechansim for _generating_ them on demand).

As one overly specific example:

The following is a set of assets that each contain 4096 nodes, arranged in a grid. The only difference between these assets is the number of _meshes_. The number of meshes is `[1, 8, 64, 512, 4096]`, meaning that in the first case, all nodes refer to the _same_ mesh, and in the last case, each node refers to a _different_ mesh:

[output-numMeshes-2024-02-18.zip](https://github.com/KhronosGroup/glTF-Sample-Assets/files/14322991/output-numMeshes-2024-02-18.zip)

The meshes are still equal: They have no textures, no materials, and each has only one mesh primitive for a plane with 10x10 points, and all these mesh primitives share the same accessor data (otherwise, all this wouldn't fit into 130KB)

---

An aside: The visual appearance is boring...

![Khronos Performance Test Model 001](https://github.com/KhronosGroup/glTF-Sample-Assets/assets/5597569/21dbaade-0b6f-4fe5-8a2e-09fbd42cdec5)

And even though "looking cool" is not the goal of this sort of model, that doesn't mean that such models _can_ not look cool. For example, the same model with 4096 nodes and 4096 meshes, but this time with 32 different materials each with a different texture would look like this:

![Khronos Performance Test Model 002](https://github.com/KhronosGroup/glTF-Sample-Assets/assets/5597569/3402fc20-94d1-4c66-8b8e-6ba437936deb)

There are still many dimensions fixed here. For example, all textures have the size 1024x1024, and all meshes still have a single primitive plane with 10x10 points. Arbitrary combinations of that would look similar, but could pose other challenges for engines.

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked issues and the glTF-Sample-Assets pull request discussion, then review the attached generated model examples. The open question is which performance dimensions are useful and whether models should be generated on demand. Done requires an agreed test-model scope and generation approach.

Written by the indexing model from the issue text.

Assessment

Domain
computer-graphics, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.