security-union / security-union/videoeditor
render: parallelize frame capture across a Chrome tab pool (frames are pure functions of t)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Summary
Frame capture is strictly sequential today, but the engine's own contract makes every frame independent: templates are pure functions of (data, t) (enforced by design — no CSS animations, no timers, __sceneSeek(t) is idempotent). That makes rendering embarrassingly parallel, and on an M-class laptop we're leaving most of the cores idle during the slowest part of the pipeline.
Where the time goes
crates/videoeditor/src/render.rs (render_web_scene):
for i in 0..total_frames {
let t_ms = i as f64 / ep.meta.fps as f64 * 1000.0;
chrome.seek(t_ms)?; // Runtime.evaluate over one WS
chrome.screenshot(&frames_dir.join(...))?; // Page.captureScreenshot over the same WS
}
One Chrome (one process, one page target, one synchronous tungstenite WebSocket — crates/videoeditor-chrome/src/lib.rs) does seek → screenshot → base64-decode → fs::write for every frame, one at a time. A 1080×1920 scene at 30fps means hundreds of round-trips with zero overlap: while Chrome rasterizes + PNG-encodes, the Rust side is blocked on the socket; while Rust decodes base64 and writes the PNG, Chrome is idle. Scenes are also rendered one after another (run() loop).
Proposal: a small tab pool, frames fanned out across it
- Keep one Chrome process (launch is already guarded against profile races), but open N page targets via
/json/new— each with its own WebSocket. Chrome gives each target its own renderer process, so rasterization genuinely parallelizes. - Each worker tab does the same
navigate(template)+init_scene(data_json)once, then pulls frame indices from a shared queue (work-stealing not needed — a chunkedAtomicUsizecounter is enough since frames cost roughly the same). rayonfits naturally on the Rust side (par_iterover frame indices with a per-thread tab via a small pool), or a plainstd::thread::scopewithmin(available_parallelism, jobs)workers — either way the base64-decode +fs::writealso moves off the single thread.Chromeis!Sync(stateful WS), so it's one tab handle per worker, never shared.- Scene-level parallelism composes on top for
videoeditor render <dir>with many scenes (each scene claims a tab), but frame-level is the win that also helps the common--scene Xiteration loop.
Determinism is untouched: same (data, t) → same pixels, and output filenames f_{i:05}.png are index-derived, so capture order is irrelevant. encode_frames still runs once after the scene completes.
Details to keep in mind
sceneWarnings()should run once (on one tab), not N times, or warnings print N-fold.- Progress line (
\r frames: i/total) needs an atomic counter instead of loop order. - Pool size: cap it (
--jobs, default something likemin(cores - 2, 8)) — each renderer target costs real memory at 1080×1920, and past ~8 tabsPage.captureScreenshotthroughput tends to flatten before memory does. Worth a quick benchmark table in the PR. - Multiple Chrome processes (instead of tabs) would need the profile-dir nonce fixed first: the temp profile is keyed on
std::process::id()of videoeditor (videoeditor-chrome/src/lib.rs), so twoChrome::launches from one run would collide. Tabs avoid this entirely; if processes ever become preferable, add a per-launch nonce. video-clipscenes already bypass Chrome (ffmpeg) and are unaffected.
Expected impact
Anecdatum from a real episode (M4 Max, 1080×1920@30): a 5-scene render is a coffee break today, with 14+ performance cores never breaking a sweat. Even a conservative 4–6× on frame capture changes the director loop from "render, wait, review" to near-interactive.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in crates/videoeditor/src/render.rs at render_web_scene and inspect the Chrome handle and synchronous WebSocket implementation in crates/videoeditor-chrome/src/lib.rs. Trace navigate, init_scene, sceneWarnings, frame capture, and encode_frames before choosing a bounded tab-worker design. Done means independent frames render concurrently, warnings run once, progress remains accurate, output names stay deterministic, and a benchmark shows the impact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100