THU-MAIC / THU-MAIC/OpenMAIC

[MAIC Editor] Phase 2 — universal scene-DSL editor + AI editing (MAIC Agent)

Open
#646 1 comment 0 reactions 0 assignees View on GitHub
area:editor priority:P1 status:in-progress type:epic
Dominant language
TypeScript
Stars
37.2k
Forks
5.9k
Avg merge
1d 3h
Merged PRs (30d)
195

Description

## What MAIC Editor is (refined from Phase 1)

Phase 1 (#560) framed MAIC Editor as an editor for **static display components**, and drew a line between "static" and "interactive" scenes — putting quiz / PBL / live "out of contract." That axis was a **category error**: static-vs-interactive describes a scene's *runtime*, not whether its underlying object can be edited.

A MAIC scene is a **DSL object**. It carries more than displayed content:

- **content** — the slide canvas, the quiz's questions, an interactive widget's config, …
- **actions** — playback orchestration: narration (speech), spotlight, laser, whiteboard cues
- **structure / metadata** — order, whiteboards, multi-agent config

**MAIC Editor is the editing framework for the MAIC scene DSL.** Its reach is the whole object — every scene type, content *and* actions *and* structure. Nothing is excluded by runtime paradigm; there are only surfaces not yet built. "Slide editor" was always too narrow; "static-display editor" was the same narrowing one level up.

### What keeps "edit everything" tractable

Two load-bearing bounds carried from Phase 1 keep universality from becoming an unbounded scope:

1. **Bounded by the DSL.** "Everything" means *every field of the finite scene DSL*, not "any conceivable feature." The DSL is the contract and the boundary.
2. **The AI-first ceiling (Phase 1 principle #2 still holds).** The framework is universal *by architecture*; manual surfaces stay **minimal** and cover only what humans do better than AI. AI covers the rest. "Edit everything" is a statement about reach, not a commitment to a hand-built UI for every field.

### Two editing dimensions

- **Content surfaces** — per scene type, via the pluggable `SceneEditorSurface` contract. slide = freeform canvas (shipped); **quiz = structured form (Phase 2)**; interactive / PBL config = later surfaces. Same chrome contract; quiz is the proof it generalizes beyond canvas.
- **Actions / playback** — a cross-cutting layer every scene carries. Edited in Phase 2 via the **AI editing sidebar**, whose first capability is **(re)generating a scene's `actions` to match its current content**.

> Sibling-framework caveat, narrowed: editing that configures **interactive behavior / runtime orchestration** (PBL agents + issue board, live discussion) is genuinely a different editing task and may stay a sibling. But this is about the *editing shape*, not "interactive scenes are out" — and it is not a Phase 2 concern.

## New in Phase 2: MAIC Agent

Phase 2 introduces a second standardized module, **MAIC Agent** — a standard **TypeScript** agent-loop runtime for OpenMAIC's AI editing. Open-source OpenMAIC is a pure-TS product, so a TS runtime is the natural fit.

What it provides: an LLM tool-calling loop with streaming, a multi-provider model abstraction, tool registration for app-domain operations, and the transport to drive a chat UI. The implementation base (build from scratch vs adopt an existing runtime) is settled in the module's own design — not pinned here.

- **Domain integration:** tools that mutate scene state; a transport to the editor sidebar; quota / metering hooks.
- **v0 boundary:** only enough to power the editor's action generation. MCP, sub-agents, and consolidating other AI workloads onto this runtime are out of Phase 2.
- _(May graduate to its own RFC, mirroring RFC #547 for MAIC Editor.)_

## Phase 2 scope

### Track 1 — Structured manual editing (decoupled from MAIC Agent)

- **Slide content surface completion** — z-order to-front/to-back, slide background (solid + image), image replace / crop / flip. Surface-level only; no chrome / store changes.
- **Quiz content surface** — the second `SceneEditorSurface`: form-based editing of questions / options / answers / analysis / points. **Validates the contract carries a non-canvas, structured surface.** Edits flow through the existing scene-data update path.

### Track 2 — MAIC Agent v0 (foundation for AI editing)

- Stand up the MAIC Agent runtime (tool-calling loop + streaming + provider abstraction + domain tool registration), server-side, with a streaming transport to the editor. Implementation base settled in the module's own design.

### Track 3 — AI editing sidebar (first MAIC Agent consumer)

- A Cursor-like right-side **chat** panel, scene-context-aware (default scope: the active scene). Intent router; Phase 2 wires **only** the "(re)generate this scene's `actions`" capability — other intents are declined for now. The shell is the final form; capabilities are added incrementally.
- This single capability:
1. gives newly inserted / created scenes playable `actions`, so we **re-enable scene creation** (`SCENE_CREATION_ENABLED`, gated off in #612) — "fine-tune" becomes "authoring";
2. **resyncs narration** after a visual edit (closes the Phase 1 known limitation: editing slide text didn't update its speech);
3. cleans duplicated scenes' stale `actions` (doubled narration + dangling spotlight/laser element ids).

## Out of scope (Phase 2)

Later iterations of the same sidebar / runtime, not separate features:

- AI derivation across surfaces ("generate a quiz from this slide", "suggest distractors")
- Full per-scene **content** regeneration (#202) — Phase 2 only regenerates `actions`, a constrained slice
- Proactive inline coach (the reserved hint slot)
- Free-form chat editing of arbitrary fields
- MCP / sub-agents in MAIC Agent
- Editors for behavior-configuration scenes (PBL / live)
- PPTX import, theme / template system

## Design principles (carried from #560, axis corrected)

1. **Round-trip is non-negotiable** — every edit round-trips through the surface's export path. Slide → PPTX. **Quiz must declare its own export target** (open question — see below); a property the export can't represent can't be edited.
2. **AI-first; manual editing is the fine-tune layer** — unchanged, and now the governor of "edit everything" (see above).
3. **Pro mode = takeover** — unchanged.
4. **~~Static-display only~~ → Universal over the DSL, bounded by the AI-first ceiling** — the contract is general over surface shape (canvas + structured); no scene type is excluded by paradigm.

## Open questions (resolved in per-track implementation plans)

1. MAIC Agent's physical home (a module inside openmaic vs a separate package) and server runtime shape (Next.js route handler stream vs standalone Node service).
2. Whether open-source OpenMAIC already has a TS-side generation pipeline that emits scene `actions` — reuse its prompt contract if so; otherwise MAIC Agent introduces net-new TS-side LLM calling (a feature in its own right, to be stated in the PR narrative).
3. Whether `SceneEditorSurface` hides a canvas / Frame dependency; the quiz surface is the forcing function to clean it up.
4. Offline fallback when re-enabling scene creation: seed a minimal playable speech action when no Agent is available.
5. Quiz's export target (round-trip principle).
6. MAIC Agent's implementation base (build from scratch vs adopt an existing TS runtime) — settled in the module's own design.

## Branch strategy

Long-lived feature branch **`feat/maic-editor-v1`** off `main`; all Phase 2 sub-PRs target it; one final integration PR merges to `main`, mirroring Phase 1's `feat/maic-editor-v0` flow. The whole editor stays behind `NEXT_PUBLIC_MAIC_EDITOR_ENABLED` (default OFF).

## Sub-issues (dependency order)

- [x] #647 — **Slide content surface completion** (z-order / background / image crop+flip) — done, PR #648 merged into `feat/maic-editor-v1`
- [x] #657 — **Quiz content surface** — done, PR #661 merged into `feat/maic-editor-v1`
- [ ] **MAIC Agent v0** — TS agent-loop runtime (loop + streaming + provider abstraction + domain tools); implementation base TBD in its own design
- [ ] **AI editing sidebar** — action (re)generation + re-enable scene creation; depends on MAIC Agent v0
- [ ] **CONTRIBUTING / RFC update** — carried from #560; document the refined DSL-universal framing + surface-authoring, and update/close RFC #547

## Success criteria

- [ ] All sub-PRs land in `feat/maic-editor-v1`; CI green at every boundary (incl. round-trip as a gate)
- [ ] Quiz editable via its surface; `SceneEditorSurface` shown to carry a non-canvas surface with no canvas-specific leakage
- [ ] AI sidebar generates valid `actions` for a scene; scene creation re-enabled; inserted/created scenes are playable
- [ ] Editing slide text and resyncing narration produces matching speech
- [ ] Existing playback / generation / export unchanged with feature flag OFF
- [ ] `tsc --noEmit`, `pnpm test`, `pnpm lint`, `pnpm build` green at every boundary

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.