KhronosGroup / KhronosGroup/glTF
JSON based node description for `KHR_interactivity`
- Dominant language
- HTML
- Stars
- 7.8k
- Forks
- 1.2k
- Avg merge
- 17h 26m
- Merged PRs (30d)
- 5
Description
The pull request at https://github.com/KhronosGroup/glTF/pull/2293 adds a specification for `KHR_interactivity` in form of an AsciiDoc document. There have been considerations to additionally have a representation of the node definitions in a machine-processable form. In order to keep that PR focussed on the _specification itself_, I'll open this issue for further discussion about that topic.
Starting with disclaimers:
- What I'm posting here is **NOT** a normative reference! The ground truth can be found in the linked PR.
- What I'm posting here is not even a "proposal". Maybe not even a suggestion. Rather some random output from some experiments.
If we decide to define _some_ form of JSON-based representation, then the question of how a possible JSON-based representation and the actual specification document could be aligned (when one of them changes) still has to be decided. This includes the question of what will be the normative source of truth, given that the normative-ness can only be achieved with the `latexmath`-based _descriptions_ **and** the additional text that comes with the tables that define the actual node structures.
---
## Representations
I'm summarizing a few, first thoughts about possible representations (and the full definitions and outputs are attached below). The representations have slightly different structures, which all could make sense for one case or another. Other representations may make more sense. This is intended for discussion, and ... maybe someone will find some of that useful.
### Basic representation
One basic representation is that of a plain JSON hierarchy where the propery names reflect the section names, like
`Nodes -> Math Nodes -> Constant -> E`
The innermost elements could then be the descriptions of the actual nodes. An example:
```json
{
"name" : "Nodes",
"children" : [ {
"name" : "Math Nodes",
"children" : [ {
"name" : "Constants",
"nodes" : [ {
"title" : "E",
"name" : "math/e",
"description" : "Euler's number",
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "2.718281828459045"
} ]
},
```
In this representation, the types are still written in the form that they have in the specification document. For example, with types like `float{2|3|4}` or `floatN`:
```json
{
"title" : "Length",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float{2|3|4}",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}
```
### Spread representation
Similar to the above, but with with types like `float{2|3|4}` or `floatN` being "instantiated" accordingly - so each node that existed for `float{2|3|4}` now exists for `float2`, `float3`, `float4`. For example, for `math/length`:
```json
"name" : "Vector Nodes",
"nodes" : [ {
"title" : "Length",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float2",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}, {
"title" : "Length",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float3",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}, {
"title" : "Length",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float4",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}
```
### Spread with subsections
Similar to the above, but with all type instantations being in an own subsection. I.e. the instantations of the `math/length` nodes are now in a subection called `math/length`:
```json
"children" : [ {
"name" : "math/length",
"nodes" : [ {
"title" : "Length (float2)",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float2",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}, {
"title" : "Length (float3)",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float3",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}, {
"title" : "Length (float4)",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float4",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
} ]
}
```
---
Many other representations are possibe. Maybe the exact instantiation or section structure is also something that could or should be decided by the _consumer_ of that JSON data. In that case, we should opt for the most generic and versatile representation (i.e. the first one, from which all others can be derived).
---
### Current output
The actual sets of node definitions in these three representations is attached here:
[leztew-output-2025-02-19.zip](https://github.com/user-attachments/files/18873653/leztew-output-2025-02-19.zip)
---
#### Asides
This output is generated with some experimental project at https://github.com/javagl/leztew which I just switched to "public", because ... why not. Originally, this had a name like `KhrInteractivityProcessing`, and was just an experiment for processing AsciiDoc to begin with. However, one reason for trying out different structures was that I wanted to play a bit with feeding this data into some [Flow processing library](https://github.com/javagl/Flow) that I created a few years ago:

The leftmost column reflects the structure of the JSON representation (spread with subsections, as it can be seen in the `math/length` node).
(The other colums are just filtered views on the nodes that can provide an _input_ or consume and _output_ of the currently selected node - many degrees of freedom for how to make something like this accessible...)
Contributor guide
Research direction
Start by reviewing pull request #2293, the attached leztew-output-2025-02-19.zip, and the example representations in this issue. The work first requires deciding whether a JSON representation is wanted, which representation to use, and how it stays aligned with the AsciiDoc specification and its normative source.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- json
- Domain
- documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100