apache / apache/tvm

[Bug] [Web] Batched WebGPU dispatch (#18871) causes UI jank from GPU compositor starvation

Open
#19,342 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
13.7k
Forks
4k
Avg merge
2d 1h
Merged PRs (30d)
112

Description

## Expected behavior

The batched WebGPU dispatch introduced in #18871 should improve or maintain UI responsiveness when running compute-heavy workloads (e.g. LLM inference) in a browser tab.

## Actual behavior

Batching all compute dispatches into a single `GPUCommandEncoder` and submitting with one `queue.submit()` call monopolizes the GPU with a single large command buffer, starving the browser's compositor of GPU time. This causes visible UI jank: laggy scrolling, frozen CSS animations, and unresponsive input in the browser tab running the workload.

Reverting to per-dispatch submission (one encoder + submit per dispatch) eliminates the jank entirely.

### Video demonstration

This recording shows the CSS animation bar stuttering during the batched dispatch phase but running smoothly during per-dispatch: https://www.loom.com/share/6832f44692f14c948020c65e6941ced7

### Benchmark data (Apple M5, Chrome)

**Throughput** — batched is marginally faster, as expected:

| Strategy | Median (ms) |
|---|---|
| Per-dispatch submit | ~600 |
| Batched submit | ~595 |

**UI Responsiveness** (`requestAnimationFrame` timing) — batched introduces frame spikes:

| Strategy | Mean frame (ms) | P95 frame (ms) | P99 frame (ms) | Worst frame (ms) | Janky (>33ms) |
|---|---|---|---|---|---|
| Per-dispatch submit | 8.29 | 8.90 | 9.30 | 9.40 | 0 |
| Batched submit | 8.35 | 9.20 | 9.40 | **166.60** | **1** |

The jank is intermittent — sometimes it shows up in rAF timing (as above), other times it's only visible in the CSS animation. This is because the stutter occurs at the GPU/compositor level: the batched command buffer delays the compositor's rendering work, but the JS main thread remains unblocked (it's awaiting `onSubmittedWorkDone`), so rAF callbacks may still fire on schedule even when frame *presentation* is delayed.

## Environment

- **OS:** macOS (Apple M5 MacBook Pro)
- **Browser:** Chrome (latest stable)
- **TVM:** main branch at HEAD (includes #18871)

## Steps to reproduce

1. Open the [benchmark HTML](https://gist.github.com/gnguralnick/8e27069f25a285833ed7288e366cea55) in Chrome
2. Watch the blue CSS animation bar at the top of the page
3. Click "Run Benchmark"
4. Observe the animation during "Benchmarking per-dispatch submit..." (smooth) vs "Benchmarking batched submit..." (stutters)

The benchmark dispatches N compute kernels (default 200) with configurable GPU load, comparing per-dispatch submit vs batched submit. The HTML file is self-contained with no dependencies.

## Analysis

#18871 batches all compute dispatches into a single `GPUCommandEncoder`, flushing only on sync/readback. The companion Metal PR (#18877) demonstrated 1.14–1.95x throughput gains on M4 Max with the same approach.

However, the WebGPU and Metal cases differ in a key way: the Metal PR (#18877) inlines blit encoders for copies into the same command buffer, keeping everything in a single submission without breaking the pipeline. The WebGPU PR (#18871) cannot do this — `deviceCopyToGPU` uses `queue.writeBuffer()` (a separate queue operation), and copies create separate encoders. More importantly, in a browser context, a single large GPU command buffer prevents the compositor from interleaving its own rendering work between dispatches, causing the UI jank described above.

A simple workaround is to call `flushCommands()` after every dispatch (effectively reverting to per-dispatch submission), which eliminates the jank. A more nuanced solution might involve periodic flushing every N dispatches to balance submission overhead against compositor starvation. Not sure what the desired behavior is, so filing this as an issue rather than making a PR. Happy to try out making a PR for whatever is the best solution. The current behavior makes a downstream use I have of web-llm, where I use an LLM to process text on the page as I scroll, pretty unusable because of how laggy the scrolling becomes.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the WebGPU batching introduced in #18871, including flushCommands() and deviceCopyToGPU, then reproduce the comparison with the linked self-contained benchmark in Chrome. Evaluate per-dispatch and periodic flushing against UI responsiveness and throughput; done means eliminating compositor jank without an unacceptable performance regression.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
performance, web-dev
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.