Perceived speed: streaming has no pacing, and here is the inventory of cadence knobs that gate responsiveness
- Dominant language
- Rust
- Stars
- 41k
- Forks
- 3.6k
- Avg merge
- 13h 59m
- Merged PRs (30d)
- 299
Description
## The complaint
Streaming reads as bursty and inconsistent rather than as typing, and the TUI
generally feels laggy in a way that is hard to place. Reported by the founder
against Codewhale (the TUI), not the GPUI app. This issue records what was found
and what still needs measuring; it is a starting inventory, not a finished
profile.
## Finding 1: the stream display coalesces but does not pace
`docs/MOTION_CONTRACT.md` states the design plainly: a commit beat
unconditionally flushes everything received since the previous beat. The clock
bounds *how often* the transcript is mutated, never *how fast* it is uncovered.
- `crates/tui/src/tui/streaming/mod.rs` -- `DEFAULT_STREAM_COMMIT_INTERVAL` 16 ms.
`StreamBuffer::take` drains the whole buffer; `StreamingState::commit_text`
hands the whole thing to the transcript.
- `crates/tui/src/tui/ui/frame.rs` -- `commit_streaming_display_tick` runs at
most one beat per interval and appends whatever accumulated.
Consequence: the visible text inherits the provider's burst shape exactly. A
200-byte SSE chunk that lands after a 300 ms pause shows as a 200-byte jump.
There is nothing between "provider sent a lump" and "user sees a lump".
**This is a deliberate prior decision, and reversing it is a product call.**
v0.9.4 deleted an adaptive-chunking policy and a `LineBuffer` newline gate, on
the grounds that the policy could only ever decide "drain everything available".
That is a fair criticism of *that* implementation, but the tree now has no
pacing at all, so the criticism does not survive as a reason to leave it that
way.
## Finding 2: the drain budget advances text in lumps of 16 events
`crates/tui/src/tui/ui/terminal_input.rs`:
MAX_ENGINE_EVENTS_PER_DRAIN = 16
ENGINE_DRAIN_TIME_BUDGET = 8 ms
Each loop iteration drains at most 16 engine events or 8 ms of drain work.
Under a sustained burst, visible text therefore advances in fixed 16-event lumps
at the loop cadence regardless of how the provider actually paced its output.
## Finding 3: the cadence inventory
| Knob | Value | Location |
| --- | --- | --- |
| Idle poll | 48 ms | `ui.rs` `UI_IDLE_POLL_MS` |
| Active poll (loading/streaming/agents) | 24 ms | `ui.rs` `UI_ACTIVE_POLL_MS` |
| Low-motion poll | 96 / 120 ms | `ui/terminal.rs` |
| Stream commit beat | 16 ms | `streaming/mod.rs` |
| Draw cap | 8.33 ms (120 FPS) | `frame_rate_limiter.rs` `MIN_FRAME_INTERVAL` |
| Draw cap, low motion | 33.3 ms | `LOW_MOTION_MIN_FRAME_INTERVAL` |
| Atmosphere interval, no probed Hz | **120 ms** | `display_refresh.rs` `FALLBACK_ANIMATION_MS` |
| Ghostty atmosphere / interactive | 30 Hz / 60 Hz | `display_refresh.rs` |
Two entries are worth a second look on their own: the 120 ms atmosphere
fallback when the terminal's refresh rate cannot be probed, and the 30 Hz
Ghostty atmosphere tier.
## Finding 4: the reported tradeoff
Smooth pacing and minimum latency pull against each other. Uncovering a burst
gradually means showing it later. The GPUI client hit the same wall and was
fixed there (commit `9ad39d7`) by revealing at a bounded rate with the backlog
target as the ceiling, which took the largest single-frame reveal from 91 bytes
to 19 and cut the spread of per-frame advancement by roughly three quarters.
The TUI needs the same decision made deliberately, with a rate that keeps up
with a fast model rather than one that reads as slow typing.
## What is NOT established
No provider delta arrival timestamps were captured against frame times. Until
that exists, Findings 2 and 3 are read from the code rather than measured, and
it is not known how much of the perceived burstiness is provider cadence, SSE
buffering, the drain budget, or the beat. That measurement is the next step and
it does not require a person to sit and watch.
log delta arrival timestamps vs advance() frames during one real turn,
then compare against the TUI rendering the same stream.
Contributor guide
Research direction
Start with docs/MOTION_CONTRACT.md and the streaming paths in crates/tui/src/tui/streaming/mod.rs, crates/tui/src/tui/ui/frame.rs, and crates/tui/src/tui/ui/terminal_input.rs; capture provider delta arrival timestamps against advance() frames during one real turn. Compare the resulting cadence with the listed poll, draw, and atmosphere limits, then define the pacing decision and measurements that show the perceived burstiness is improved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100