OpenCut-app / OpenCut-app/OpenCut

Rewrite input: a captions + karaoke data model that survived production (and where it fought us)

Open
#898 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
89.8k
Forks
8.9k
PR merge metrics
No merged PRs in 30d

Description

Context: we ship a production product (FrameOS) built on the classic OpenCut web editor (MIT — thank you). On top of it we built a full caption system — visual style picker, transcript import, word-level karaoke highlights, per-range restyling — and shipped it across ~10 releases. Since the rewrite is plugin-first and will need a first-class captions story, here's the data model that survived production and the five places it fought us, which together are an argument for captions as a typed core primitive rather than a convention. (Filed as architecture-phase input per the README; related: #811.)

The model that worked

Captions as a group of ordinary timeline elements: every element of a caption run carries a captionGroupId plus a captionRole (chunk — the visible text block — or highlight — the karaoke word overlay). Word-level karaoke is just per-word overlay elements on their own track, each one's window running to the next word's start. Style edits fan out group-wide with role-aware rules (e.g. color/background apply to chunks only, because a highlight's color is the karaoke; size changes rescale highlight offsets proportionally).

Building on plain elements bought us rendering, transforms, dragging, and keyframe animation for free.

Where it fought us — the case for a first-class primitive

  1. Split: the generic split command silently no-ops on elements not containing the playhead, so "split captions at playhead" (a bread-and-butter captioning action) needed special-casing.
  2. Drag: retiming a caption chunk must carry its karaoke companions; group-move semantics had to be bolted onto the interaction controller.
  3. Replace vs. coexist: "add captions" must find existing caption runs to replace — stamped groups, legacy caption-named tracks, word-shaped highlight tracks. Identity-by-convention is exactly as fragile as it sounds.
  4. Timeline rendering: caption cues want to render as one continuous ribbon of flush segments (the Resolve/Premiere convention — zero-inset, hairline separators). We tried collapsing them into a single quiet bar and reverted: it broke split and selection, because the bar wasn't the thing the commands operated on.
  5. Restyling: mutating styled elements in place lost information. What worked: stamp a captionSource (pre-preset text + word timings) on insert, and rebuild cues from it on restyle. Selection is the styling scope: one cue selected = whole-group fan-out; several = range-only.

Two robustness notes from production incidents

  • Caption sizes must come from the render/export spec, not preview-space values. Ours looked fine in the picker and rendered invisible on canvas until the two agreed.
  • Font loading must be non-fatal. FontFace.load() rejects with a stack-less DOMException("NetworkError") when, e.g., an ad-blocker eats a Google Fonts request — and if project load awaits fonts inside its fatal path, a cosmetic failure kills the whole project open. Fonts are cosmetic; degrade, don't abort.

Distilled suggestion for the rewrite

Make caption group + role + rebuild-source a typed primitive in the core model (with karaoke windows as derived data), and give split/drag/replace group-aware semantics there — rather than free elements plus conventions. A plugin API then gets captions "for free", and every editor UI (desktop, mobile, browser) renders the same ribbon.

Happy to elaborate on any of these, including the exact fan-out rules and the failure modes we hit.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start with the rewrite README and related issue #811, then trace the existing timeline element, split, drag, replace, rendering, and restyling entry points. Done means defining a typed caption group with roles and rebuild-source data, plus group-aware semantics for split, drag, and replace.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.