KhronosGroup / KhronosGroup/glTF

JSON based node description for `KHR_interactivity`

Open
#2,472 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
HTML
Stars
7.8k
Forks
1.2k
Avg merge
17h 26m
Merged PRs (30d)
5

Description

The pull request at https://github.com/KhronosGroup/glTF/pull/2293 adds a specification for `KHR_interactivity` in form of an AsciiDoc document. There have been considerations to additionally have a representation of the node definitions in a machine-processable form. In order to keep that PR focussed on the _specification itself_, I'll open this issue for further discussion about that topic.

Starting with disclaimers:

- What I'm posting here is **NOT** a normative reference! The ground truth can be found in the linked PR.
- What I'm posting here is not even a "proposal". Maybe not even a suggestion. Rather some random output from some experiments.

If we decide to define _some_ form of JSON-based representation, then the question of how a possible JSON-based representation and the actual specification document could be aligned (when one of them changes) still has to be decided. This includes the question of what will be the normative source of truth, given that the normative-ness can only be achieved with the `latexmath`-based _descriptions_ **and** the additional text that comes with the tables that define the actual node structures.

---

## Representations

I'm summarizing a few, first thoughts about possible representations (and the full definitions and outputs are attached below). The representations have slightly different structures, which all could make sense for one case or another. Other representations may make more sense. This is intended for discussion, and ... maybe someone will find some of that useful.

### Basic representation

One basic representation is that of a plain JSON hierarchy where the propery names reflect the section names, like
`Nodes -> Math Nodes -> Constant -> E`
The innermost elements could then be the descriptions of the actual nodes. An example:
```json
{
"name" : "Nodes",
"children" : [ {
"name" : "Math Nodes",
"children" : [ {
"name" : "Constants",
"nodes" : [ {
"title" : "E",
"name" : "math/e",
"description" : "Euler's number",
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "2.718281828459045"
} ]
},
```

In this representation, the types are still written in the form that they have in the specification document. For example, with types like `float{2|3|4}` or `floatN`:

```json
{
"title" : "Length",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float{2|3|4}",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}
```

### Spread representation

Similar to the above, but with with types like `float{2|3|4}` or `floatN` being "instantiated" accordingly - so each node that existed for `float{2|3|4}` now exists for `float2`, `float3`, `float4`. For example, for `math/length`:

```json
"name" : "Vector Nodes",
"nodes" : [ {
"title" : "Length",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float2",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}, {
"title" : "Length",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float3",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}, {
"title" : "Length",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float4",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}
```

### Spread with subsections

Similar to the above, but with all type instantations being in an own subsection. I.e. the instantations of the `math/length` nodes are now in a subection called `math/length`:

```json
"children" : [ {
"name" : "math/length",
"nodes" : [ {
"title" : "Length (float2)",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float2",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}, {
"title" : "Length (float3)",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float3",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
}, {
"title" : "Length (float4)",
"name" : "math/length",
"description" : "Vector length",
"inputValueSockets" : [ {
"type" : "float4",
"name" : "a",
"description" : "Vector"
} ],
"outputValueSockets" : [ {
"type" : "float",
"name" : "value",
"description" : "Length of stem:[a], e.g., stem:[sqrt(a_x^2 + a_y^2)] for `float2`; see the description for details"
} ]
} ]
}
```

---

Many other representations are possibe. Maybe the exact instantiation or section structure is also something that could or should be decided by the _consumer_ of that JSON data. In that case, we should opt for the most generic and versatile representation (i.e. the first one, from which all others can be derived).

---

### Current output

The actual sets of node definitions in these three representations is attached here:

[leztew-output-2025-02-19.zip](https://github.com/user-attachments/files/18873653/leztew-output-2025-02-19.zip)

---

#### Asides

This output is generated with some experimental project at https://github.com/javagl/leztew which I just switched to "public", because ... why not. Originally, this had a name like `KhrInteractivityProcessing`, and was just an experiment for processing AsciiDoc to begin with. However, one reason for trying out different structures was that I wanted to play a bit with feeding this data into some [Flow processing library](https://github.com/javagl/Flow) that I created a few years ago:

![Image](https://github.com/user-attachments/assets/2e673ea2-85ef-404b-9abe-91d7521d58d4)

The leftmost column reflects the structure of the JSON representation (spread with subsections, as it can be seen in the `math/length` node).

(The other colums are just filtered views on the nodes that can provide an _input_ or consume and _output_ of the currently selected node - many degrees of freedom for how to make something like this accessible...)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing pull request #2293, the attached leztew-output-2025-02-19.zip, and the example representations in this issue. The work first requires deciding whether a JSON representation is wanted, which representation to use, and how it stays aligned with the AsciiDoc specification and its normative source.

Written by the indexing model from the issue text.

Assessment

Tech stack
json
Domain
documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.