godotjs / godotjs/GodotJS

RFC: optional embedded TypeScript transpiler (SWC), build-flag-gated

Open
#225 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
765
Forks
46
PR merge metrics
No merged PRs in 30d

Description

# RFC: Optional embedded TypeScript transpiler (SWC)

**Status:** proposal for discussion. With the flag off (the default), no Rust enters the build graph and every `.ts` load/resolve path is gated out — the default build is functionally unchanged.

> Note for maintainers: this is a draft prepared by the hatgg team (the fork that runs this in production). "We"/"hatgg" below = that team, not the GodotJS maintainers.

## Summary

Add an **optional, build-flag-gated** embedded TypeScript transpiler — SWC, compiled in as a small Rust staticlib — so GodotJS can load `.ts` directly and produce source-mapped stack traces, without requiring `tsc`. **Off by default:** with the flag off, no Rust toolchain is needed, the `.ts` runtime/resolver paths are compiled out behind `JSB_WITH_TYPESCRIPT_TRANSPILER`, and the engine builds and behaves as it does today (script loading keeps using the `tsc`-emitted `.js`). Prepared as three independently-reviewable PRs (the first is ~540 lines of C++/Rust plus a committed `Cargo.lock`).

## Motivation

Today a `.ts` file must be compiled to `.js` by `tsc` before the engine can run it. That means a separate build step, a dependency on a `tsc` install, and — in the GodotJS editor panel — an install gate on the **"Start tsc (watch)"** button. Embedding a fast transpiler:

- removes the build step from the edit→run loop (load `.ts` directly), and
- enables **`.ts`-accurate stack traces** at runtime via inline sourcemaps — the thing that makes a transpiled language actually debuggable.

## Non-goals

- **Not** removing or replacing the existing `tsc` workflow. It stays, coequal and default.
- **Not** making Rust a default build requirement. The default (flag-off) build is unchanged.

## Design (built to answer the obvious objections)

- **Opt-in flag** `use_typescript_transpiler=no` (default). Off → no Rust toolchain needed; the cargo build, the staticlib link, and every `.ts` load/resolve path are all gated by the flag, so the default build is functionally unchanged.
- **Desktop-only runtime loader.** The in-engine transpiler builds only on platforms with a mapped Rust target triple (macOS/Linux/Windows). web/android/ios have no triple: they compile the load path out and ship export-time pre-transpiled `.js` (see PR 3). With the flag on but no triple (e.g. `use_typescript_transpiler=yes platform=web`), SCsub prints a warning and builds with the transpiler compiled out.
- **Tiny FFI surface:** two `extern "C"` functions + one `#[repr(C)]` struct, hand-written (not cbindgen). **Panic-safe:** a `catch_unwind` at the boundary turns any transpiler panic into an error result instead of unwinding into C++. Buffers cross the boundary as `Box<[u8]>` so the C-side free is sound.
- **Pinned & reproducible:** `Cargo.lock` committed (194 crates); `[profile.release] lto=true, codegen-units=1`; vendorable for offline/reproducible builds. (Caveat: `rustc` itself is not yet pinned — see Costs.)
- **Build-time *or* runtime:** the transpiler can be used purely at **export time** (pre-transpile, zero runtime engine dependency) or as a **runtime loader** — and the runtime path is itself behind the flag. The most conservative adoption is export-time-only.

## Backwards compatibility

- **Additive, and gated.** With the flag off, existing `tsc`-based projects build and run exactly as today (the script loader and resolver keep resolving `.ts` references to the `tsc`-emitted `.js`).
- **The opt-in is per *build*, not per project.** A binary built with the flag *on* loads `.ts` sources through SWC for every project it opens; the project's `tsconfig.json` emit options (`target`, `useDefineForClassFields`, `experimentalDecorators`, `emitDecoratorMetadata`) are **not** consulted — the embedded pipeline is fixed at `es2022` + CommonJS + TC39 standard decorators. Teams that want `tsc`'s exact emit keep the flag off.
- **Known limitations under the SWC path** (stated up front, not discovered later):
- SWC lowers **TC39 Stage-3 standard decorators only** (`decorator_2022_03`), which is the convention GodotJS's recommended `createClassBinder()` runtime is built for. Projects that explicitly enable legacy `experimentalDecorators` **and** use the deprecated string-name `@export_`/`@signal` annotations fail fast under SWC with a clear runtime error (those APIs are already `@deprecated`; the migration is `createClassBinder()`). `emitDecoratorMetadata` is unsupported.
- Ambient `declare const enum` is not inlined (a single-file transpiler can't see other files' constants — same as esbuild/babel/`isolatedModules`). Same-file `const enum` *is* inlined.
- **Parity, as a proposed merge gate.** The stack proposes a `tsc`-vs-SWC conformance suite under `tests/project`: each fixture run twice (the `tsc`-emitted `.js` vs SWC-from-`.ts`), asserting identical observable behavior over the surface GodotJS relies on — the modern `createClassBinder()` decorator surface (class/field/method/accessor/signal/rpc), auto-accessors, same-file and exported `const enum`, `es2022` class-field define semantics, and `import x = require()`. The limitations above are asserted as clean errors, not parity. (hatgg has verified the positive cases empirically against the crate; the suite formalizes them.)

## Incremental delivery (3 PRs; each is independently correct with the flag off)

1. **Embed transpiler + load `.ts`** (flagged). The Rust staticlib + FFI + build/link plumbing + the flag-gated `.ts` load path. Flag off ⇒ no behavior change.
2. **Inline sourcemaps + `Error.stack` remap.** Real `.ts` stack traces. (Note: also reworks `Error.stack` remapping through V8's prepare-stack-trace hook for the *existing* `tsc` sourcemap path on V8 builds — see that PR's body.)
3. **Export-time pre-transpile + drop the hard `tsc` gate.** Exported builds need no `tsc`; the no-transpiler resolver resolves a `.ts` reference to its pre-transpiled `.js` sibling so web/mobile run the packed `.js`; the panel's "Start tsc (watch)" button stops hard-blocking when `tsc` isn't installed.

## Costs (stated plainly)

- Flag-on builds need `cargo` on `PATH`; the first build compiles the SWC dependency tree (several minutes with `lto=true, codegen-units=1`, then incremental).
- A new ABI seam (mitigated: tiny, panic-safe, `Box<[u8]>` ownership transfer, pinned lock).
- **Link flags, flag-on only:** a flag-on build links with `-Wl,--allow-multiple-definition` (Linux) / `/FORCE:MULTIPLE` (Windows) to tolerate `libstd` symbols (`rust_eh_personality` and a few others) duplicated between our staticlib and Godot's prebuilt `accesskit` staticlib (the clash source is editor-only in Godot, so the editor is the realistic case). These relax duplicate-symbol detection binary-wide; an open question below proposes a more surgical alternative.
- **`rustc` is not pinned** (only crates are). Two contributors on different `rustc` versions can get different output, and the duplicate-symbol set above depends on the toolchain. A `rust-toolchain.toml` would fix this.
- Maintenance: the hatgg team commits to owning the component, a flag-on CI lane, and SWC version bumps.

## Open questions for maintainers

1. Acceptable as **opt-in / off-by-default**?
2. Preference for landing order: **export-time-only first** (no runtime dependency), or the runtime loader first?
3. Do you want the **`tsc`-parity conformance suite** as a required merge gate?
4. Web/WASM: the web target is export-time-only **by design** — in-engine web transpile would require linking the Rust tree into the Emscripten build (unsolved, not planned). Is export-time pre-transpile acceptable as the permanent web story?
5. Prefer the `libstd` duplicate-symbol clash handled by a pinned `rust-toolchain.toml`, or by post-processing the staticlib (localize/strip `libstd` symbols), so the binary-wide link flags can be dropped?

## Future possibilities (explicitly out of scope — what the seam makes possible later)

None of this is proposed here; it is why an *embedded* compiler (vs. shelling out to an external `swc`/`esbuild`) is worth the seam. Each would be gated behind the reserved `opts` FFI flag, with no semantic divergence by default.

- **Compile-time codegen passes.** A spike (branch `rpc-codegen-spike`, `transpiler-ffi/tests/rpc_codegen_spike.rs`, test green against `swc_core` 37) rewrites an `__rpc(name, mode)` marker into `jsb.internal.add_script_rpc(...)` at transpile time — RPC/signal boilerplate generated by the compiler instead of the decorator runtime.
- **JSX/TSX.** SWC's `swc_ecma_transforms_react` crate is already in the pinned dependency tree; enabling it is a feature flag, not new dependencies.

## Status of the implementation

Built as a three-commit stack off current `main` (each PR is one commit, reviewable against the previous; opened sequentially, or as one combined PR since the tip's diff is the whole stack). Each commit is independently correct with the flag off. A tiny panic-safe FFI, behind the off-by-default `use_typescript_transpiler` flag, with changesets, internally reviewed at hatgg. The **export-time** path is in production — hatgg's shipped web titles are exported this way (`.ts` pre-transpiled at export, the packed `.js` resolved and run at runtime in-browser, verified end-to-end). The **runtime `.ts` loader** is in daily editor use across the hatgg team — the environment it targets, since SWC never ships in an exported game. Ready to open when there's appetite — or to be reshaped (e.g. export-time-only) based on this discussion.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.