Progressive glTF: Exploring Ideas

Hi!

For 2d images on websites, it is well known that so called progressive jpegs provide the best possible user experience. What are these? They allow the browser to display a low-resloution placeholder almost immediately, while the image is still loading. It gets more refined as the rest of the image file is loaded.

I have similar hopes for glTF. Our use case would be to display placeholders while the file still loads from the web or from the hard drive to the GPU. This is important: Even loading uncompressed data from SSD to GPU can take a while, especially for large textures and complex meshes, so I'm interested in this even for files even if they are already downloaded.

There are already a few things you can do to achieve a similar effect right now. Some ideas would be possible with extensions. I'm curious what you think, let's discuss :)

## Definitely Possible:
- load json first and display bounding box placeholders, maybe for individual nodes (probably only glTF, not glb)
- load meshes and display them before loading textures
- use multiple gltf/glb files and load the smallest first, replacing previous placeholders
- load some nodes before other nodes (which ones to load first? maybe largest bounding box size first? this might be easier when standardized with a non-required extension?)

## Maybe possible?
- use progressive jpeg textures and already display the low-resolution version while still loading the remaining texture? is it allowed in the specification?

## Extensions/Extras?
- `texture.extensions.previewColor` to display a solid color instead of the texture while it is still loading? Like a 1x1 mip map? or maybe even 8x8 values directly in the json?
- include an LOD mechanism that allows the runtime to choose one of multiple meshes?
- maybe a generic placeholder extension that applies to many objects (nodes, meshes, textures, ...) that can be used as an approximation? it would represent the same data but a smaller copy, for example an 8x8 png or 200 vertex mesh or a simplified node with an sinplified mesh for multiple children?
- `node.extensions.allowProgressive` to indicate that this node hierarchy is allowed to be displayed partially while still loading? `node.extensions.showChildrenAsLoaded` to indicate that the first child is the most important one?

## Other Questions
Are there more possibilities? Are there ways to load a few bytes of a glb file and already display some data while the file is still being transmitted? Are the glb contents ordered?

In general, when prioritizing all heavy binary data, what order to load then would be the most beneficial? In other words, which binary resources are the most visually important? Which parts can be loaded in parallel? Is it possible to determine a general heuristic which data to load first? All geometry and then all textures, or some nodes before others? Are there states that are not useful to display?

Maybe, compute a visual importance factor and a predicted loading time for each binary asset, and then sort according to that? The visual importance factor could be a combination of bounding box size, data type (geometry probably being more important than textures), whether it is animated, whether it is rather transparent, and similar properties.

I'm excited to hear your ideas! :) Is anyone else seeing value in this?

No specific file, test, or entry point is named. Start by reviewing the glTF specification and the extension ideas discussed in the issue; a concrete proposal, agreed scope, and implementation target would be needed before this work can be considered done.

Written by the indexing model from the issue text.