github / github/copilot-cli

Keep a persistent append handle for `events.jsonl` instead of open/append/close per event (Windows Defender rescan / CPU cost)

Open
#4,063 0 comments 2 reactions 0 assignees View on GitHub
area:platform-windows area:sessions
Dominant language
Shell
Stars
11.2k
Forks
1.9k
Avg merge
14h 16m
Merged PRs (30d)
6

Description

### Summary

Each session's event log at `~/.copilot/session-state//events.jsonl` is appended one event at a time via `fsPromises.appendFile(path, line, { mode })` (guarded by a per-file `runExclusive` mutex). When `appendFile` is called with a **path** (rather than an open `FileHandle`), Node opens the file with flag `'a'`, writes, and **closes** it on every call — i.e. a `CreateFile → WriteFile → CloseHandle` cycle **per emitted event**.

On Windows, that `Create`+`Close` per event is what triggers Windows Defender (and other real-time AV) to rescan the file — the close, not the write, is the scan trigger. During an active, tool-heavy turn events are emitted tens–hundreds of times per second, so Defender rescans `events.jsonl` (which grows monotonically through the turn) that many times per second, producing a sustained CPU/energy regression on the user's machine.

We're seeing this downstream in a desktop app (Microsoft Scout) that bundles the Copilot CLI; `events.jsonl` is the largest remaining source of AV rescans after we cut our own state-file write churn. Tracked internally as ADO 62904486, and possibly the same root cause as #3907 ("CPU usage at max.").

### Where

- Append: `runExclusive(lockKey, async () => { await mkdir(dir, { recursive: true }); await appendFile(eventsPath, line, { mode }); })` — one open/append/close per event.
- Compaction: the `truncate` path does a single `writeFile(eventsPath, fullContent, { mode })` (full rewrite). That's occasional and fine.

### Ask

Hold a **persistent append `FileHandle`** open per session and reuse it:

- Open once per session with `fs.open(eventsPath, 'a')` (or a cached `createWriteStream`), then append via `fileHandle.appendFile(line)` / `fileHandle.write(line)` for each event, and `close()` once at session end (or on the `truncate`/compaction rewrite, then reopen).
- Because the handle stays open, the file identity is preserved and there is no per-event `Create`/`Close`, so AV real-time protection incrementally scans only the appended bytes instead of rescanning the whole file on every event.
- The existing `runExclusive` mutex already serializes writers, so a single long-lived handle is compatible — the only added lifecycle concern is closing/reopening around the `truncate` rewrite and on session teardown.

If a persistent handle is undesirable, a smaller mitigation is to **batch events** and flush on a short interval / at turn boundaries, which reduces (but doesn't eliminate) the open/append/close frequency.

### Impact

- Removes the dominant per-event AV rescan trigger on Windows; the cost currently scales with turn length × event volume, so long tool-heavy turns — when the machine is already busy — are the worst case.
- Append-only semantics and on-disk format are unchanged; `events.jsonl` remains the documented hook/event stream (cf. #3551).

### Environment

- OS: Windows 10/11, Microsoft Defender real-time protection enabled.
- Copilot CLI (`@github/copilot`) bundled inside an Electron desktop app.
- Path: `%USERPROFILE%\.copilot\session-state\\events.jsonl`.
- Observed mechanism: `fsPromises.appendFile(path, …)` per event (verified in the bundled `app.js`).

### Downstream reference

gim-home/m#4426 — same class of problem (Defender rescans from high-frequency small-file writes) for the desktop app's own `.scout` state files, mitigated there by coalescing writes; `events.jsonl` is the CLI-owned remainder.

Contributor guide

Open the contributing guide

Research direction

Find the session event-log append implementation by searching for fsPromises.appendFile(eventsPath) and its runExclusive lock. Trace session teardown and the truncate or compaction path to understand when a handle must close or reopen. Done means events.jsonl keeps its existing format while appends reuse a persistent handle and lifecycle behavior remains safe.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.