Roadmap: scale Try Playwright toward 1M concurrent users

Target

Treat Try Playwright as a pet distributed system and actually scale it.

Load to design for: ~1M concurrent users, ~3 runs/user/minute → ~50k new runs/s. If a run lasts ~15s, that is ~750k concurrent executions.

Today we are nowhere near that: one control replica, exclusive RabbitMQ reply queues, in-memory SSE, warm one-shot pods (~1 CPU / 1Gi, RestartPolicy: Never). That model is correct for isolation, and cannot hit this load. The work is to keep isolation while moving cost from “a Kubernetes pod per snippet” to “a dense, recycled sandbox plus a horizontally scaled control plane.”

Related: live logs (#320 / log-watch SSE) pin streams to one control process. That has to go before control HPA is real.

Non-goals (until the steps below)

• Multi-region (do this last).
• Live screenshot streaming.
• Perfect fairness across languages on day one.

Steps

1. Name the bottleneck, then split the control plane from the request path

Why: handleRun still borrows a pod, talks AMQP, and (for SSE) holds state in process. Exclusive reply queues already mean control cannot HPA. K8s Pods.Create on the user path will not do 50k/s.

Do: Control becomes a stateless API: auth, admit, enqueue runId, return 202. Logs/SSE read a shared stream. A pool manager (not the API) creates/deletes sandboxes.

Pro	API replicas scale; failures don’t drop every in-flight AMQP consumer.
Con	Two extra moving parts (queue + run store).

2. Shared run log bus (replace in-memory hub + exclusive AMQP replies)

Why: Live logs and multi-replica control need the same story: runId → log lines → done.

Do: Redis Streams or NATS JetStream keyed by runId (job payload + log/done). SSE gateways XREAD/subscribe that key. Keep RabbitMQ only if we outgrow “simple” before we outgrow one broker box—don’t run two brokers.

Pro	Any replica can serve /log-watch; control HPA works.
Con	New ops surface; must TTL/trim streams or they become a disk bomb.

3. Admission, queues, and fairness (before more hardware)

Why: 50k/s of Chromium is a budget problem first. Unbounded enqueue is how you melt the cluster.

Do: Global + per-user token buckets; max in-flight; queue with delay SLO (e.g. p99 queue wait 2s, else 429). Shed load at the edge (Caddy/CDN) before Go.

Pro	Predictable cost; protects workers.
Con	Users wait or get 429; need a clear UX.

4. Dense workers, not one k8s pod per run

Why: 750k pods is not a cluster, it’s a career. Isolation still matters (arbitrary code).

Do, in order:

1. Warm pool + KEDA on current one-shot pods — good to ~hundreds of concurrent runs, teaches autoscaling. Not the 1M design.
2. Long-lived VM/microVM sandboxes (Firecracker/gVisor/Kata), recycle per run (wipe FS, kill leftover browsers). Pool sized by in-flight runs, not by “API replicas.”
3. Pack N slots per node with hard cgroup + network egress via the proxy mesh.

Pro	10–100× density vs pod-per-run; still throw away the world after each snippet.
Con	Biggest engineering chunk; recycling bugs = escape/cross-talk.

Skip “process-level multi-tenant Node” without a VM boundary at this scale.

5. Autoscale every stateless tier the same way

HPA/KEDA on:

• frontend/Caddy (static + reverse proxy; CDN the assets)
• control API (after 1–2)
• SSE/log gateways (separate from POST /run so a million EventSources don’t block admission)
• file-service → become a thin validator; presign PUT/GET to object storage, don’t stream all bytes through one Go deploy
• egress proxy (Squid is a single choke point): fleet of proxies, or eBPF/envoy egress, with per-run identity so one user can’t eat the NAT

etcd for shares will not take this write rate → Redis/S3 for snippet blobs.

Pro	Each piece scales on its own signal (QPS, stream count, egress bandwidth).
Con	More services to page on; need one load-test harness or you scale the wrong replica.

6. Multi-region last

Why: Browsers + artifact uploads hate one continent. Only pays off after a dense worker pool exists.

Do: Regional worker+proxy+log-bus; global admission (or regional tokens); run id stays in-region.

Pro	Latency and blast radius.
Con	Consistency for share URLs; 3× ops.

Suggested order of attack

1 → 2 → 3 (control actually scales, load can be refused) → 5 on edge/files/proxy (cheap wins) → 4 (the real capacity) → 6.

How we know we’re not lying

A load profile, not vibes: N concurrent SSE clients, R runs/s, queue wait p99, $/run. First milestone can be 100 concurrent runs with HPA control; the 1M-user math is the north star so we don’t “autoscaling” a pod-per-run control loop and call it done.

Start by reading the handleRun path and the /log-watch endpoint, then inspect how the control API, RabbitMQ replies, in-memory SSE state, and one-shot Kubernetes pods are connected. Use the suggested order 1 → 2 → 3 as the initial scope rather than attempting the entire roadmap. Done should be demonstrated with a load profile covering concurrent SSE clients, runs per second, queue-wait p99, and cost per run.