google-gemini / google-gemini/gemini-cli

perf(core): chat history O(n^2) re-serialization on every turn causes jank on long sessions

Open
#29,080 3 comments 1 reaction 0 assignees View on GitHub
area/core effort/medium kind/bug priority/p2 status/bot-triaged status/need-information
Dominant language
TypeScript
Stars
107k
Forks
14.6k
Avg merge
2d 3h
Merged PRs (30d)
45

Description

### What happened?

In `packages/core/src/services/chatRecordingService.ts` and `packages/core/src/core/geminiChat.ts` (main @ 812f7a2bc), conversation history handling has O(n^2) performance and memory issues that degrade severely on long sessions (100+ turns, common with agents):

`geminiChat.ts` lines ~1180-1195:
```ts
getHistoryTurns(curated: boolean = false): HistoryTurn[] {
const history = curated
? extractCuratedHistory(this.agentHistory.get())
: [...this.agentHistory.get()];

if (this.context.config.isContextManagementEnabled()) {
return scrubHistory(history);
}

const model = this.context.config.getModel();
if (isGemini2Model(model) || supportsModernFeatures(model)) {
return coalesceConsecutiveRoles(stripThoughts(history));
}

return history;
}
```

`chatRecordingService.ts`:
```ts
updateMessagesFromHistory(history: HistoryTurn[]) {
// Re-serializes entire history on every turn
const serialized = JSON.stringify(history, null, 2);
fs.writeFileSync(this.sessionFile, serialized);
}

recordMessage(message: HistoryTurn) {
this.history.push(message);
this.updateMessagesFromHistory(this.history); // O(n) write per message
}
```

Problems:
1. **O(n^2) on history growth**: Every `recordMessage` and `updateMessagesFromHistory` serializes and writes the *entire* history to disk. For a session with 500 turns (each ~2KB), this is 1MB JSON written 500 times = 500MB total I/O, with each write taking longer as history grows.
2. **No incremental append**: Should append single message to file (JSONL) rather than rewriting entire file
3. **Synchronous file I/O**: `writeFileSync` blocks the event loop for 10-50ms per turn on large histories, causing UI jank
4. **Memory duplication**: `getHistoryTurns` clones history via `[...this.agentHistory.get()]` then `scrubHistory` clones again via `structuredClone`, holding 3x memory (original + curated + scrubbed) for large histories (100 turns * 10KB = 1MB * 3 = 3MB, but with tool outputs can be 10x larger)
5. **No history compaction**: Old tool outputs (e.g., `read-file` of large files) are kept forever in `comprehensiveHistory` even though `curatedHistory` filters them — but `chatRecordingService` stores comprehensive, not curated

### What did you expect to happen?

- Use append-only JSONL or incremental updates to session file
- Use async `writeFile` with debouncing (e.g., batch writes every 100ms)
- Cache `getHistoryTurns` results and invalidate only when history changes
- Implement history compaction: truncate old tool outputs beyond context window, or store curated history for persistence

### Client information

- Source-level finding verified against upstream `main` at commit `812f7a2bc`
- Files: `packages/core/src/services/chatRecordingService.ts`, `packages/core/src/core/geminiChat.ts:1172-1195`
- Affects all platforms, especially long agent sessions

### Login information

Not applicable.

### Anything else we need to know?

Sources:
- https://github.com/google-gemini/gemini-cli/blob/812f7a2bc/packages/core/src/services/chatRecordingService.ts
- https://github.com/google-gemini/gemini-cli/blob/812f7a2bc/packages/core/src/core/geminiChat.ts#L1172-L1195
- Repro: Run 200-turn agent session (e.g., "refactor this large codebase"), observe `~/.gemini/tmp//conversation.json` grows to 5MB, each turn takes 50-100ms longer than previous, UI lags

Searched existing issues for "history performance", "chatRecording performance", "O(n^2) history" — no open duplicate found.

Contributor guide

Open the contributing guide

Research direction

Read packages/core/src/services/chatRecordingService.ts and packages/core/src/core/geminiChat.ts around lines 1172-1195, then reproduce the behavior with a 200-turn agent session. Trace history serialization, file writes, and history cloning before choosing and documenting a bounded approach. Done means long sessions no longer show progressively increasing per-turn latency or unnecessary history duplication, with focused coverage for the changed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
cli, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.