Proposal: EXT_WASM_script extension

# Proposal: `EXT_WASM_script`

A Vendor-Agnostic WebAssembly-Based Scripting Extension for glTF

## Overview

This proposal introduces a new extension, **`EXT_WASM_script`**, that enables attaching scripts written in any language compiled to **WebAssembly (WASM)** directly to glTF assets. By defining a common WASM Component interface for glTF, implementers can embed Turing-complete behaviors in a vendor-neutral, platform-agnostic manner.

This work draws inspiration from [KHR_interactivity](https://github.com/KhronosGroup/glTF/pull/2293) but opts to leverage existing WASM tooling instead of defining a new node-based, vendor-agnostic programming system from scratch.

**Accessibility**
1. **Language Choice**: By targeting WASM, developers can author interactive scripts in their favorite language (C, C++, Rust, AssemblyScript, etc.), removing potential language-barrier or specialized skill-set hurdles.
2. **Widespread Platform Support**: WASM is a W3C standard supported across platforms (browsers, servers, embedded devices), so the same script can be deployed everywhere glTF is used, helping ensure broad accessibility.

**Ubiquity**
1. **Standardized Runtime**: The ubiquitous presence of WASM runtimes (in browsers, game engines, server frameworks, etc.) means minimal friction for glTF viewers to support this extension.
2. **Modular Composition**: The emerging [WASM Component Model](https://component-model.bytecodealliance.org/) standardizes module composition, further easing adoption by allowing interoperable building blocks across multiple platforms and runtimes.

---

## Existing Work

### [KHR_interactivity](https://github.com/KhronosGroup/glTF/pull/2293)

The [KHR_interactivity](https://github.com/KhronosGroup/glTF/pull/2293) proposal introduces a node-based approach to embedding programmatic logic in glTF. Although it is a powerful idea (effectively making glTF Turing-complete), it requires a substantial effort from vendors to map or export existing node graphs (e.g., Unreal Engine Blueprints) to the proposed glTF format.

### WebAssembly (WASM)

**WASM** is a binary instruction format standardized by the W3C for a stack-based virtual machine. It is designed as a portable compilation target, enabling widespread adoption on the web and beyond. Projects like the [Bytecode Alliance](https://bytecodealliance.org/) have championed WASM on multiple platforms and are developing the [WASM Component Model](https://component-model.bytecodealliance.org/) for composable modules with platform-agnostic interfaces.

---

## Problem to Solve

Interactivity or scripting in a 3D format often relies on platform-specific extensions or custom engines. While `KHR_interactivity` aims to unify this with a node-based system, exporting and translating existing codebases (especially large ones in various programming languages) into such a new node graph format can be a significant burden.

**Key needs include**:
- A widely supported binary execution environment that can be used by all glTF viewers
- The ability to write interactive logic in a wide range of programming languages
- A standard interface for hooking user inputs (e.g., keyboard, pointer) and runtime events (e.g., per-frame updates) into the logic controlling the 3D scene

---

## Proposal

### Introducing `EXT_WASM_script`

This extension allows glTF files to reference **WASM modules** that implement a standardized interface for interactivity. A possible example interface, using WIT (the WebAssembly Interface Type syntax), could look like:

Example: WIT Interface for glTF Interactivity

```wit
// A hypothetical WIT package for glTF Interactivity
package gltf_interactivity: "0.1.0"

// The 'world' describes how the host and the component connect.
world gltf_interactivity {
// The host calls these exported APIs to pass data/events to the Wasm component.
export gltf_interactivity-inputs: gltf_interactivity_inputs
}

// An interface for providing input data and events to the Wasm component.
interface gltf_interactivity_inputs {
/// Dispatch a pointer event (mouse/touch).
/// For example: move, down, up, click, etc.
pointer_event: func(event: PointerEvent)

/// Dispatch a keyboard event (key press or release).
keyboard_event: func(event: KeyboardEvent)

/// Let the component know a frame/tick is happening (time-based update).
/// Typically called once per frame, with delta-time in seconds.
tick: func(delta_time: float32)
}

// ----- Records and Enums for Events -----

// Example pointer event data
record PointerEvent {
event_type: PointerEventType,
x: float32, // canvas or viewport x
y: float32, // canvas or viewport y
}

// Pointer event "type" enumeration
enum PointerEventType {
Move,
Down,
Up,
Click,
// Additional events as needed...
}

// Example keyboard event data
record KeyboardEvent {
key_code: u32, // e.g., ASCII/UTF-8 code, or a platform-specific scan code
is_down: bool, // true if pressed, false if released
}
```

Under the hood:

1. **Authoring**
- Write interactive logic in any WASM-capable language (C, Rust, AssemblyScript, etc.).
- Export it to a WASM module that implements the chosen interface (e.g., for handling pointer events, keyboard events, etc.).
- Reference the WASM module from the glTF using the `EXT_WASM_script` extension.

2. **Runtime**
- A glTF viewer loads the WASM module in a WASM VM.
- It validates the module’s compliance with the expected interactivity interface.
- It connects user input events (e.g., pointer or keyboard) and frame updates to the WASM module’s exported functions.

---

## Benefits

1. **Easy Integration of Existing Code**
Many libraries (physics engines, AI modules, domain-specific logic) already compile to WASM. This extension lets developers drop in widely used solutions without rewriting them for a custom node system.

2. **Vendor Neutral**
The specification focuses on hooking into a standardized WASM interface. There is no mandated vendor toolchain, meaning broad adoption can be more frictionless.

3. **Broad Language Support**
Authors can choose any language with WASM support, which already includes C/C++, Rust, Go, AssemblyScript, and more. This fosters inclusivity for both seasoned and new developers.

4. **Performance & Security**
WASM is sandboxed, providing security benefits in embedded systems or untrusted code environments. It also executes with near-native performance in many scenarios.

5. **Cross-Platform Ubiquity**
Virtually every major platform (desktop, mobile, web, VR, AR, IoT devices, servers) can execute WASM modules, making interactive glTF content consistently deployable across diverse targets.

---

## Future Enhancements

- **WASM [Component Model](https://component-model.bytecodealliance.org/) Adoption**
As the [Component Model](https://component-model.bytecodealliance.org/) matures, additional interfaces (beyond input events and ticking) can be standardized (e.g., physics, audio, networking), making glTF even more interactive and extensible.

- **Shared Memory / Bindings**
Further standardization of communication channels or shared data structures between the glTF scene graph and WASM modules (such as scene node transforms, materials, etc.) could deepen integration.

- **Extended Event Types**
Beyond basic pointer/keyboard events, future expansions might include VR/AR controllers, advanced haptics, or platform-specific input devices.

- **Built-In Editor Tooling**
Tools like Blender, Unity, or Unreal Engine might eventually provide direct exporters that embed or bundle WASM scripts, streamlining the authoring pipeline for interactive glTF experiences.

---

## Conclusion

**`EXT_WASM_script`** proposes leveraging the ubiquity and flexibility of WebAssembly to deliver accessible, high-performance scripting in glTF. By aligning with the WASM standard and enabling multiple programming languages, we reduce adoption barriers and simplify the integration of existing codebases. This approach offers a complementary path to the node-based KHR_interactivity, granting developers an additional, readily available tool to bring dynamic, interactive content to glTF.

**We welcome feedback** from the Khronos community on how best to formalize and refine this extension.

**Thank you for reading and considering this proposal!**

No repository files, tests, or implementation entry points are named; start by reviewing the proposal and its relationship to KHR_interactivity. Done would require a concrete, agreed extension specification and identified repository changes, which this issue does not yet define.

Written by the indexing model from the issue text.