SOTA experiment: reproduce browser layer sharding for RuVector WASM and LatentMesh

Finding

SwarmLLM published a public MIT licensed browser runtime and a September 7, 2026 demo that splits Qwen 3.8 27B across browser tabs and sends an approximately 10 KB hidden state per generated token over direct WebRTC links. The originating team reports 400 tokens at 10.7 tok/s across a MacBook and iPhone on the same WiFi. It also reports a GB10 WebGPU decode result of 9.0 tok/s plain and 16.1 tok/s with speculative decoding versus 8.0 tok/s for llama.cpp on the same quantized model, while native prefill remains much faster.

Primary source: https://github.com/Nehanth/swarmllm

Evidence class: originating team report with public code and public benchmark logs, not independently reproduced by RuV.

License: MIT according to the upstream repository. Pin and independently verify the exact reviewed commit before any code reuse.

Significance

This is potentially more important for RuV than another single device WebGPU kernel optimization. It demonstrates a plausible browser native model parallel primitive across heterogeneous consumer hardware with direct peer transport. If reproduced, it can combine RuVector WASM execution with LatentMesh transport and RuFlo placement for local sovereign inference on devices that individually cannot hold the model.

It is not yet evidence for production private inference. Upstream explicitly notes that room peers can observe prompts, outputs, and activations and that activations are not a privacy boundary against a determined peer.

Opportunity score

Weighted RuV Opportunity Score: 4.000 / 5.

Applicability 4.5, performance impact 5.0, implementation speed 3.0, cross stack leverage 4.5, commercial value 4.0, strategic differentiation 5.0, security improvement 2.5, open source leadership 4.0, evidence confidence 2.5, long horizon option value 5.0, reversibility 4.5.

Proposed experiment

Do not fork the full runtime first. Reproduce the core primitive as a bounded adapter experiment:

1. Partition one already supported quantized transformer into contiguous layer ranges.
2. Execute each range through a pinned RuVector WASM/WebGPU worker.
3. Transport only the required activation payload between workers through a transport interface with local loopback, WebRTC data channel, and LatentMesh-compatible adapters.
4. Keep tokenizer, KV ownership, prompt state, and sampling ownership explicit in the benchmark manifest.
5. Compare plain pipeline decode, speculative decode if available, and single-device baseline.
6. Preserve a local-only mode so federation or external relay availability cannot affect the benchmark.

The reusable primitive is a LayerShardTransport plus placement manifest, not a new model format.

Baselines

A. Current single-device RuVector WASM/WebGPU path on hardware that can fit the model.

B. llama.cpp using the same GGUF or semantically equivalent quantization on the same hardware where possible.

C. Upstream SwarmLLM pinned commit and documented model as an external reproduction baseline.

D. A loopback multi-shard path on one device to isolate orchestration and serialization overhead from the network.

Required environments

At least:

• one Apple Silicon desktop or laptop with current Chrome
• one iPhone or iPad Safari path if supported by the candidate
• one Chromium desktop on a non-Apple GPU if available
• one same-LAN two-device run
• one impaired network run with controlled latency, jitter, packet loss, and peer loss

Record browser build, OS, WebGPU adapter, GPU memory, system memory, device thermals where available, quantization, model digest, exact layer allocation, WebRTC settings, and power source.

Metrics

Report baseline and candidate with seeds and sample sizes:

• time to first token
• prefill tokens per second
• decode tokens per second
• p50, p95, and p99 inter-shard transfer latency
• activation bytes per token and total network bytes
• peak GPU and system memory per peer
• energy per generated token where measurable
• speculative acceptance rate
• quality equivalence and deterministic golden-output checks where supported
• peer join, leave, and failure recovery time
• output corruption rate under loss, reordering, duplicate frames, malformed frames, and peer restart
• privacy exposure surface and whether raw prompts, outputs, KV state, or activations are visible to each peer

Acceptance

Accept the primitive for further integration only if all of the following hold:

1. Independent reproduction matches upstream decode throughput within 20% on at least one documented environment or explains the gap with measured hardware/browser differences.
2. Multi-device decode is at least 20% faster than the strongest device that can run the chosen model alone, or enables a model that no individual participating device can fit while remaining above 5 tok/s interactive decode.
3. Inter-shard transport adds no more than 20% p95 decode latency on same-LAN runs.
4. Golden-output or tolerance-based equivalence detects no unexplained numerical drift.
5. Peer failure produces a bounded stop or safe replan, never silent corrupted continuation.
6. Malformed or unauthenticated peer frames cannot mutate model state outside the declared shard protocol.
7. No claim of confidential inference is made without a separate cryptographic or trusted execution design.

A negative result is valuable. If network synchronization or browser memory behavior dominates, record the rejection and keep the single-device WASM path.

Security and governance

• Treat every remote peer as untrusted until authenticated and explicitly admitted.
• Model shard ownership and peer identity do not grant RuFlo or RVM authority.
• Do not transmit tenant prompts, outputs, or activations outside the declared trust domain during production experiments.
• Add per-peer byte, time, memory, and compute budgets.
• Reject model or shard digest mismatch before execution.
• No autonomous merge, deployment, external credential use, or model download outside the pinned benchmark manifest.

Cross stack mapping

RuVector WASM: WebGPU layer execution, quantized weights, browser workers, kernel profiling.

LatentMesh: activation transport, adaptive pacing, degradation and peer-failure experiments.

RuFlo and Autogenous: placement, peer capability discovery, bounded failover, cost and liveness receipts.

MetaHarness: independent reproduction, fault injection, quality and performance gates.

RVM and RVF: execution authority remains separate; receipts can bind model digest, shard assignment, peer identity, and benchmark evidence.

Cognitum: possible local sovereign inference product for homes, facilities, and edge fleets without requiring a single large GPU.

Next action

Freeze an upstream commit, model digest, browser builds, and two-device hardware pair in MetaHarness before writing the RuVector adapter. Do not import upstream code until license and dependency review are attached.

Start by freezing the SwarmLLM upstream commit, model digest, browser builds, and hardware pair in MetaHarness before writing the RuVector WASM/WebGPU adapter. Compare single-device, loopback, and multi-device runs using the listed metrics and fault cases. Done means the acceptance thresholds are measured, security constraints are addressed, and a negative result is recorded if synchronization or browser memory dominates.