[FEATURE] Headless Rendering SDK + Visual Config UI for programmatic video generation

Problem

OpenCut today is a polished in-browser editor, but it lacks a programmatic entry point. This single gap caps both the user base and the strategic positioning of the project:

• AI agents cannot drive OpenCut. With LLM function calling and structured output rapidly maturing (GPT, Claude, etc.), "AI auto-editing" is now a high-frequency workflow — script generation → asset/voice synthesis → automated cuts → final video. Today this still requires a human dragging a timeline.
• OpenCut cannot be embedded into other products. SaaS backends (e-commerce auto-generating product videos, news outlets bulk-producing clips, edtech turning lessons into video), CMS / design tools (Figma / Notion gaining "export as video"), game/livestream pipelines (auto-cut highlights) — none of these can integrate OpenCut without spinning up a real browser.
• Locked into the JS ecosystem. Python AI agents (LangChain, LlamaIndex), Go / Java / Rust backends, shell scripts, CI pipelines — none of them can call OpenCut today. The reachable audience is bounded by "people who can use a browser."
• No standard template format. The internal SerializedProject isn't well-suited as an LLM structured-output schema or as a community-shareable template format, so there's no foundation for a template ecosystem.
  The net result: today's AI video stack (Pika / Runway / Sora) is all black-box generation. The controllable, editable, reproducible "edit layer" that agent pipelines actually need is missing — and OpenCut is uniquely positioned to fill it, but only with a programmatic API.

Solution

Add two new packages to the monorepo, plus a thin opt-in bridge layer inside apps/web. The rendering pipeline itself is not modified.

1. @opencut/render-sdk (new package, publishable to npm)

• Library API: import { render } from '@opencut/render-sdk' for Node / TS projects.
• CLI: opencut-render run draft.json -o out.mp4 for cross-language consumers (Python / Go / Java via subprocess).
• Core: spawns a Next.js standalone server + drives headless Chromium via Playwright. Asset transfer uses page.route to intercept fetch('/opencut-asset/<alias>') — no extra HTTP server, no CORS, no OPFS gymnastics.
• RenderServer long-running class for batch workloads — a single 30s/1080p render drops from ~90s cold to ~30s warm.

2. @opencut/config-ui (new package)

• Zero-dependency local visual config page (single-file HTML + Node HTTP server).
• Authors the DraftTemplate JSON consumed by render-sdk, with one-click render preview.
• Lets non-developers still produce templates that agents / SDKs can consume.

3. apps/web/src/automation/ (added inside OpenCut, opt-in)

• types.ts — DraftTemplate + zod schema (the public-facing template format).
• compiler.ts — translates DraftTemplate → SerializedProject.
• bridge.ts — exposes window.__opencutAutomation, only injected when NEXT_PUBLIC_AUTOMATION=1 at build time, so default builds are completely unaffected.

Architecture

┌─────────────────────────────────────────────────────────────┐
│ packages/render-sdk/   (npm: @opencut/render-sdk)            │
│   Library API + CLI + Playwright-driven headless rendering   │
├─────────────────────────────────────────────────────────────┤
│ packages/config-ui/    (visual draft-template editor)        │
├─────────────────────────────────────────────────────────────┤
│ apps/web/src/automation/  (opt-in via NEXT_PUBLIC_AUTOMATION)│
│   DraftTemplate ↔ SerializedProject + window bridge          │
└─────────────────────────────────────────────────────────────┘

Sample DraftTemplate

{
  "name": "demo",
  "fps": 30,
  "resolution": { "width": 1920, "height": 1080 },
  "background": { "type": "color", "value": "#000000" },
  "assets": {
    "intro": "/abs/path/intro.mp4",
    "bgm":   "/abs/path/bgm.mp3"
  },
  "timeline": [
    { "type": "video", "clips": [{ "asset": "intro", "start": 0, "trim": [0, 3] }] },
    { "type": "audio", "clips": [{ "asset": "bgm",   "start": 0, "duration": 3, "volume": 0.5 }] }
  ]
}

Supported: video / image / audio / text / subtitle tracks, effects / masks / animations / transform, full SubtitleTrack style overrides.

Why this design

Decision	Choice	Rationale
Browser ↔ Node channel	page.route intercept	No extra server, no CORS, in-browser code is plain fetch
Bridge injection	NEXT_PUBLIC_AUTOMATION=1 opt-in	Default builds untouched; zero impact on existing users
Template format	Custom DraftTemplate (zod)	Stable, LLM-friendly, usable directly as a structured-output schema
Rendering reuse	Calls existing editor.renderer.exportProject()	Output is byte-for-byte identical to UI exports

Config UI

Expected impact

• AI agents can produce videos as a function call (DraftTemplate doubles as a function-calling tool definition).
• SaaS / CMS / local tools (Raycast, Alfred, Obsidian plugins, etc.) can integrate via a one-line CLI call.
• Cross-language reach — Python / Go / Java / shell users become first-class.
• Local-first — assets never leave the machine, a key differentiator vs. cloud alternatives (Remotion, Shotstack).
• Template ecosystem — a standardized schema makes community-contributed template packs viable.
• Zero impact on existing users — gated behind a build flag, rendering pipeline untouched.

Alternative

N/A — this is a purely additive capability with no conflict against existing features, so there is no alternative implementation to weigh against.

(OpenCut's template marks this field as required, so a brief "N/A" is included to satisfy validation.)

Anything else?

A few open questions I'd love maintainer input on before opening PRs:

1. Is this direction welcome upstream? If yes, I'll split into incremental PRs (bridge layer first, then render-sdk, then config-ui). If you'd rather it stay external, I'll keep the fork and publish under an independent npm scope.
2. Where should the apps/web/src/automation/ bridge layer live? Currently injected via layout.tsx when NEXT_PUBLIC_AUTOMATION=1. Open to a dedicated entry point or a separate next config if you have a preferred pattern.
3. DraftTemplate vs. SerializedProject — keep two layers (my preference), or unify?

I have a working prototype on a local fork (76/76 unit tests passing, end-to-end render verified) and am happy to share a draft PR, recorded demo, or both — whichever format is most useful for the discussion.

Thanks for considering! 🙏

Review the proposed apps/web/src/automation/types.ts, compiler.ts, and bridge.ts entry points, then compare their DraftTemplate/SerializedProject boundary with existing editor.renderer.exportProject(). The issue reports 76/76 unit tests and an end-to-end render on a local fork; done would require maintainer agreement on scope and incremental PR boundaries for the bridge, render-sdk, and config-ui.