JetBrains / JetBrains/thinkrail

First 'New chat' is slow because pi runtime is lazily initialized (not pre-warmed at boot)

Open
#353 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
TypeScript
Stars
478
Forks
37
Avg merge
1d 52m
Merged PRs (30d)
143

Description

Description

The first "New chat" after launching ThinkRail (desktop app or CLI) takes several seconds to open. Subsequent new chats open instantly. This is caused by lazy initialization of the in-process pi runtime: getPiRuntimeGeneration() in packages/server/src/agent/piRuntime.ts is only triggered on the first session/model request, and the first call does a full ModelRuntime.create() + extension/skill reload, which is expensive.

Steps to reproduce

  1. Launch the ThinkRail desktop app (or thinkrail CLI and open a workspace).
  2. Wait for the UI to be fully loaded.
  3. Click "New chat".
  4. Observe the delay before the chat input appears.

Note: step 3 is slow only on the first new chat after launch; the second new chat opens immediately.

Expected behavior

The chat should open immediately (or the pi runtime should be pre-warmed at boot so the first interaction is not delayed).

Actual behavior

The first new chat blocks for seconds while the pi runtime boots. From the code:

  • packages/server/src/agent/piRuntime.tsgetPiRuntimeGeneration() lazily creates the runtime on first use; createRuntimeWithExtensions() runs advanceExtensionCacheGeneration() (2× DefaultResourceLoader.reload()), ModelRuntime.create(), createAgentSessionServices() with JITI_REBUILD_FS_CACHE=1, then a further buildResourceLoader() reload.
  • packages/server/src/host/boot.tsbootHost() never touches the pi runtime (no pre-warm call).
  • The pre-warm hooks already exist (configurePiRuntimeGenerationInitializer, preparePiRuntimeGeneration) but are not wired into boot.

So the "wait" is the first createSession() / model.list paying for a full runtime bootstrap that could have been done in the background at app launch.

Environment

  • OS: macOS 26.6.2 (arm64)
  • Bun version: 1.3.14
  • Node version: v24.16.0
  • ThinkRail version / commit: 0.1.0-nightly.33 (also present in v0.0.19)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in packages/server/src/agent/piRuntime.ts by tracing configurePiRuntimeGenerationInitializer and preparePiRuntimeGeneration, then inspect packages/server/src/host/boot.ts to see how bootHost can invoke the existing pre-warm hooks. Reproduce the first-chat delay in the desktop app or thinkrail CLI; done means the pi runtime is prepared during launch and the first new chat no longer waits for runtime initialization.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, cli, desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.