pingdotgg / pingdotgg/t3code

[Bug]: One slow foreground health probe tears down a live session and disables the composer

Open
#7,231 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
23k
Forks
5.9k
Avg merge
11h 14m
Merged PRs (30d)
357

Description

Before submitting
  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.
Area

packages/contracts or packages/shared

Steps to reproduce
  1. Open the desktop app (or the web client) against a local environment.
  2. Give the backend real work: a fleet of subagents, a large repository, or any thread busy enough to keep the server's event loop saturated for a few seconds.
  3. Switch to another window and come back, which raises an application-active wakeup.
  4. Watch the environment banner and the composer.

There is no deterministic repro. It needs the backend to be slow enough that one health probe misses its deadline, which on a loaded Windows host happens often.

Expected behavior

A backend that is busy but alive should stay connected. The WebSocket is still open and no close event arrived, so the client has positive evidence the transport is fine. One slow answer is not a disconnect.

Actual behavior

The environment drops to "Failed to connect. Reconnecting..." with the reason " did not respond to a connection health check", and the composer is disabled until it recovers. The backend was never gone.

The path is in packages/client-runtime/src/connection/supervisor.ts:

  • supervisor.ts:34CONNECTION_PROBE_TIMEOUT = "15 seconds".
  • supervisor.ts:421 — on an application-active wakeup the supervisor runs lease.session.probe.
  • supervisor.ts:428 — on timeout it fails with ConnectionTransientError({ reason: "timeout" }).
  • supervisor.ts:451 — that exit is propagated, monitorConnectedLease returns, and the lease is replaced.

A single timeout is enough. There is no second look, and the code does not distinguish a deadline from a definite failure even though it constructs the two differently. The socket state is never consulted.

The user-visible damage is mostly the composer. Losing a few seconds of connection would be tolerable; losing the ability to type is not, and it lands exactly when someone has come back to the app to send a message.

Impact

Major degradation or frequent failure

Version or commit

main @ 4c1d99d7f

Environment

Windows 11, T3 Code Nightly desktop, local environment. The supervisor is shared by web, desktop, and mobile.

Logs or stack traces
Banner text:

  Failed to connect. Reconnecting...
  Reason: <environment label> did not respond to a connection health check.
Screenshots, recordings, or supporting files

No response

Workaround

Wait. It reconnects on its own, usually within seconds. There is nothing to do in the meantime because the composer is disabled.

Related

#3553 reported this and was closed by #4137, which made the probe itself lightweight (serverProbe instead of serverGetConfig). That was the right fix for probe cost, and it helps, but it does not address the teardown rule: one timeout still replaces a live session. On a host that is starved rather than merely doing extra work, a cheap probe misses its deadline too.

#5198 proposed tolerating a transient timeout. It has been sitting with merge conflicts since 1 Aug.

Backend starvation is tracked separately in #4773; this issue is about the client's reaction to it, which is worth fixing on its own because the client cannot assume the server will ever be fast.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in packages/client-runtime/src/connection/supervisor.ts, especially CONNECTION_PROBE_TIMEOUT and the application-active path around lines 421-451. Trace how lease.session.probe timeouts become ConnectionTransientError and how monitorConnectedLease replaces the lease; compare the socket state and the prior discussion in #5198. Done means a single slow health response does not tear down a still-open session or disable the composer, while definite connection failures still recover.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.