CesiumGS / CesiumGS/cesium

Qualified names for metadata access in custom shaders

Open
#10,085 1 comment 0 reactions 0 assignees View on GitHub
category - metadata category - model/gltf
Dominant language
JavaScript
Stars
15.7k
Forks
3.9k
Avg merge
4d 6h
Merged PRs (30d)
34

Description

Right now, metadata is passed to the custom shaders via an `fsInput.metadata` structure that contains the properties. The shader code may access this data, with something like

vec2 example = fsInput.metadata.exampleVector;

where `exampleVector` is the name of a property in "the" metadata class.

When there are *multiple* classes, and two of them have a property with the same name, then this cannot be represented with this structure. (Right now, it will causes the shader compilation to bail out with some ~"duplicate field in struct" error message).

---

It will be necessary to disambiguate these names. And this disambiguation also has to take into account the _schema_ (because there might be two schemas with the same class name, where the classes use the same property name).

A straightforward solution **could** be to refer to the variables with fully qualified names, as

vec2 example = fsInput.metadata.exampleSchemaId.exampleClassName.examplePropertyName;

(where `exampleSchemaId` is the [`schema.id`](https://github.com/CesiumGS/3d-tiles/blob/fb04c0c170c0bba53507dd05d5af0c9dd6d29d65/extensions/3DTILES_metadata/schema/schema.schema.json#L13)). This is complicated on the implementation side, because it requires the creation of some deeply nested `struct`s. Alternatively, the qualification could happen with `_` underscore characters, as in

vec2 example = fsInput.metadata.exampleSchemaId_exampleClassName_examplePropertyName;

In both cases, accessing these properties becomes a bit clumsy. But they both could be reasonable ways of _having_ the data in the shader, _consistently_ and _unambiguously_.

---

``

A shader that uses data similar to the [example from the specification README](https://github.com/CesiumGS/glTF/tree/3d-tiles-next/extensions/2.0/Vendor/EXT_mesh_features#property-textures) currently looks as follows:

```
var houseTextureShader = new Cesium.CustomShader({
lightingModel: Cesium.LightingModel.UNLIT,
fragmentShaderText: [
"void fragmentMain(FragmentInput fsInput, inout czm_modelMaterial material)",
"{",
" int inside = fsInput.metadata.insideTemperature;",
" int outside = fsInput.metadata.outsideTemperature;",
" float insulation = fsInput.metadata.insulation;",
" material.diffuse = vec3(float(inside)/255.0, float(outside)/255.0, insulation);",
"}",
].join("\n"),
});
```

Adding the full qualification would yield
```
var houseTextureShader = new Cesium.CustomShader({
lightingModel: Cesium.LightingModel.UNLIT,
fragmentShaderText: [
"void fragmentMain(FragmentInput fsInput, inout czm_modelMaterial material)",
"{",
" int inside = fsInput.metadata.exampleSchema.buildingComponents.insideTemperature;",
" int outside = fsInput.metadata.exampleSchema.buildingComponents.outsideTemperature;",
" float insulation = fsInput.metadata.exampleSchema.buildingComponents.insulation;",
" material.diffuse = vec3(float(inside)/255.0, float(outside)/255.0, insulation);",
"}",
].join("\n"),
});
```

---

Not being deeply involved in the details of the custom shader mechanisms, I wondered whether it could make sense to offer some configurability on this on the API level. **Roughly** like
```
houseTextureShader.defineInput("insideTemperature", "exampleSchema.buildingComponents.insideTemperature");
houseTextureShader.defineInput("outsideTemperature", "exampleSchema.buildingComponents.outsideTemperature");
houseTextureShader.defineInput("insulation", "exampleSchema.buildingComponents.insulation");
```
so that the respective properties can then be accessed with `fsInput.outsideTemperature`. Possible (idealistic) advantages of something like this could be that

- it would be possible to (verbatim) use the same shader code, regardless of how things are named in the metadata (establishing the "wiring" between the metadata and shader inputs could happen on the API level)
- it **might** even be possible to implement that in a way that allows switching these inputs _at runtime_, without having to re-compile the shader (but there might be reasons why this is technically not viable)

``

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the custom shader metadata handling and the linked 3DTILES_metadata schema, then compare the fully qualified, underscore-qualified, and API-mapped input approaches described here. Done means selecting and implementing a consistent, unambiguous way to access properties from multiple schemas and classes, with appropriate validation or shader coverage identified during that investigation.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
computer-graphics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.