CesiumGS / CesiumGS/3d-tiles

glTF Interactivity - Behavior Graphs - 3D Tiles extensions for geospatial

Open
#809 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Batchfile
Stars
2.6k
Forks
503
Avg merge
9h 29m
Merged PRs (30d)
2

Description

glTF has explored interactivity for awhile, first considering trigger/action lists and now likely going with behavior graphs (similar to Unreal Engine Blueprints).

Ben Houston gave a talk on this to the Metaverse Standards Forum in June 2023 ([video](https://www.youtube.com/watch?v=EmpPw2VSNZs&list=PL9H8jJb7mpbmvkYoEc_k8_w8avvu8mWfT&index=34)). He developed a small open-source TypeScript library, [Behave-Graph](https://github.com/bhouston/behave-graph), to show how this can work at runtime.

The glTF Behavior Graph can be extended with custom events and actions. It would be interesting to consider what geospatial extensions would be useful for 3D Tiles, e.g., things like fly to, with a use case being Cesium Stories capability (and beyond) using visual scripting. Might be a lot of synergy across visual scripting in Omniverse, Unreal, and Unity.

Dwight Rodgers and Alexey Medvedev gave a talk on glTF interactivity in September 2024 ([video](https://www.youtube.com/watch?v=E_dRv6W3PUU&list=PL9H8jJb7mpbmvkYoEc_k8_w8avvu8mWfT&index=12)). The architecture is quite nice, decoupling the node graph itself from the glTF object model targets. We should consider:

* Extension so that 3D Tiles metadata extensions can be targeted
* Extending the node graph for more complete visual scripting, e.g., dynamic arrays, etc. - as a no-code / low-code open standards approach to building 3D geospatial experiences - basically what Blueprint is to Unreal

Note that the node graph is generic enough that it could even target things like the CesiumJS API.

How would this interact with / complement / replace 3D Tiles declarative styling? https://github.com/CesiumGS/3d-tiles/issues/2

* Latest public draft of [KHR_interactivity](https://github.com/KhronosGroup/glTF/blob/interactivity/extensions/2.0/Khronos/KHR_interactivity/Specification.adoc)
* `KHR_interactivity` pull request: https://github.com/KhronosGroup/glTF/pull/2293
* React based interactivity graph authoring tool - [glTF-InteractivityGraph-AuthoringTool](https://github.com/KhronosGroup/glTF-InteractivityGraph-AuthoringTool/tree/initial-work-merge)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the latest KHR_interactivity specification draft and its glTF pull request, then compare the proposal with 3D Tiles issue #2 on declarative styling. Review how 3D Tiles metadata could be targeted and what additional visual-scripting capabilities are needed. Done would require an agreed scope and design for the proposed extensions.

Written by the indexing model from the issue text.

Assessment

Tech stack
react, typescript
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.