[Bug] [Web] Batched WebGPU dispatch (#18871) causes UI jank from GPU compositor starvation
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 4k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 112
Description
## Expected behavior
The batched WebGPU dispatch introduced in #18871 should improve or maintain UI responsiveness when running compute-heavy workloads (e.g. LLM inference) in a browser tab.
## Actual behavior
Batching all compute dispatches into a single `GPUCommandEncoder` and submitting with one `queue.submit()` call monopolizes the GPU with a single large command buffer, starving the browser's compositor of GPU time. This causes visible UI jank: laggy scrolling, frozen CSS animations, and unresponsive input in the browser tab running the workload.
Reverting to per-dispatch submission (one encoder + submit per dispatch) eliminates the jank entirely.
### Video demonstration
This recording shows the CSS animation bar stuttering during the batched dispatch phase but running smoothly during per-dispatch: https://www.loom.com/share/6832f44692f14c948020c65e6941ced7
### Benchmark data (Apple M5, Chrome)
**Throughput** — batched is marginally faster, as expected:
| Strategy | Median (ms) |
|---|---|
| Per-dispatch submit | ~600 |
| Batched submit | ~595 |
**UI Responsiveness** (`requestAnimationFrame` timing) — batched introduces frame spikes:
| Strategy | Mean frame (ms) | P95 frame (ms) | P99 frame (ms) | Worst frame (ms) | Janky (>33ms) |
|---|---|---|---|---|---|
| Per-dispatch submit | 8.29 | 8.90 | 9.30 | 9.40 | 0 |
| Batched submit | 8.35 | 9.20 | 9.40 | **166.60** | **1** |
The jank is intermittent — sometimes it shows up in rAF timing (as above), other times it's only visible in the CSS animation. This is because the stutter occurs at the GPU/compositor level: the batched command buffer delays the compositor's rendering work, but the JS main thread remains unblocked (it's awaiting `onSubmittedWorkDone`), so rAF callbacks may still fire on schedule even when frame *presentation* is delayed.
## Environment
- **OS:** macOS (Apple M5 MacBook Pro)
- **Browser:** Chrome (latest stable)
- **TVM:** main branch at HEAD (includes #18871)
## Steps to reproduce
1. Open the [benchmark HTML](https://gist.github.com/gnguralnick/8e27069f25a285833ed7288e366cea55) in Chrome
2. Watch the blue CSS animation bar at the top of the page
3. Click "Run Benchmark"
4. Observe the animation during "Benchmarking per-dispatch submit..." (smooth) vs "Benchmarking batched submit..." (stutters)
The benchmark dispatches N compute kernels (default 200) with configurable GPU load, comparing per-dispatch submit vs batched submit. The HTML file is self-contained with no dependencies.
## Analysis
#18871 batches all compute dispatches into a single `GPUCommandEncoder`, flushing only on sync/readback. The companion Metal PR (#18877) demonstrated 1.14–1.95x throughput gains on M4 Max with the same approach.
However, the WebGPU and Metal cases differ in a key way: the Metal PR (#18877) inlines blit encoders for copies into the same command buffer, keeping everything in a single submission without breaking the pipeline. The WebGPU PR (#18871) cannot do this — `deviceCopyToGPU` uses `queue.writeBuffer()` (a separate queue operation), and copies create separate encoders. More importantly, in a browser context, a single large GPU command buffer prevents the compositor from interleaving its own rendering work between dispatches, causing the UI jank described above.
A simple workaround is to call `flushCommands()` after every dispatch (effectively reverting to per-dispatch submission), which eliminates the jank. A more nuanced solution might involve periodic flushing every N dispatches to balance submission overhead against compositor starvation. Not sure what the desired behavior is, so filing this as an issue rather than making a PR. Happy to try out making a PR for whatever is the best solution. The current behavior makes a downstream use I have of web-llm, where I use an LLM to process text on the page as I scroll, pretty unusable because of how laggy the scrolling becomes.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the WebGPU batching introduced in #18871, including flushCommands() and deviceCopyToGPU, then reproduce the comparison with the linked self-contained benchmark in Chrome. Evaluate per-dispatch and periodic flushing against UI responsiveness and throughput; done means eliminating compositor jank without an unacceptable performance regression.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- performance, web-dev
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100