Windows: long-running sessions left open across an in-place auto-update keep executing from renamed copilot.exe.old and spin one thread at 100% CPU indefinitely
- 主要语言
- Shell
- 星标
- 11.2k
- 派生
- 1.9k
- 平均合并
- 14 小时 16 分钟
- 30 天内合并 PR
- 6
描述
### Describe the bug
On Windows, a long-running interactive/`--plan` `copilot` CLI session that stays open across
an in-place auto-update keeps executing from its now-renamed `copilot.exe.old` binary
instead of exiting or picking up the new build - and a large fraction of these orphaned
processes then spin **one thread continuously at ~100% of a full CPU core**, indefinitely,
until the process is killed. On a machine with several long-lived sessions left open across
multiple update cycles, this compounds: **7 of 13** `copilot.exe` processes were found stuck
this way on one machine, each burning a full core (visible as ~6% each in Task Manager on a
16-core box), for a sustained multi-day aggregate load of roughly one full CPU core just
from stale sessions nobody was actively using.
This is very likely an under-reported contributor to "my machine feels slow after Copilot
CLI has been running a while" reports, since:
- it requires no user action to trigger (any auto-update while a session is left open),
- it's easy to miss in Task Manager (each stuck process shows a modest per-core percentage,
not an obvious 100%, once normalized across many cores),
- and it accumulates with every update cycle a session survives.
### Affected version
Observed across multiple builds: processes pinned to stale `pkg` extraction dirs `1.0.64`,
`1.0.64-3`, `1.0.66-1`, `1.0.69-0`, `1.0.69-1`, `1.0.69-2` were all still spinning while
`1.0.70` was current. Not version-specific - this looks like a structural self-update issue
rather than a regression in one build.
### Environment
- Windows 11, native (not WSL2)
- Install method: WinGet (`GitHub.Copilot_Microsoft.Winget.Source`)
- Sessions launched via Windows Terminal tabs (both directly and via a third-party sidebar
tool that shells out to `copilot.exe`)
### Diagnostic evidence
**1. Confirmed continuous single-thread spin**, not a momentary spike: re-sampled
per-thread CPU over 3 intervals (~4s each) for the 7 stuck PIDs - consistent ~3.9-4.0
CPU-seconds per 4 wall-seconds (~100% of one core) every time, with the **same thread ID**
reporting `Running` on every sample.
**2. Every stuck process is executing from a renamed `.old` binary.** Windows' own
`QueryFullProcessImageName` (queried per-PID) reports the live on-disk image name for these
processes as `copilot.exe.old`, not `copilot.exe` - i.e. these processes are provably still
running the file the auto-updater renamed out from under them
(rename-running-exe-to-`.old`, write new exe in its place is the classic in-place-update
pattern; `%LOCALAPPDATA%\copilot\pkg\win32-x64\` also contains a `.replaced--...`
directory confirming the extraction cache's own replace-on-update logic).
- Caveat for anyone reproducing this: `Get-Process -Name copilot*` (bulk/wildcard) and
`Win32_Process.Name` (WMI/CIM) both still report the **stale** "copilot" name for the
same PID even after the rename - only a targeted per-PID lookup (`Get-Process -Id
` under PowerShell 7/.NET Core, or `QueryFullProcessImageName` directly) reflects
it live. Windows PowerShell 5.1 (.NET Framework)'s `Get-Process -Id ` also shows
the stale name - only .NET Core reflects it, so a Win32 API call is the only
version-independent way to detect this.
**3. Every stuck process was pinned to a superseded, numeric-suffixed `pkg` runtime
extraction dir** (`%LOCALAPPDATA%\copilot\pkg\win32-x64\1.0.64`, `1.0.64-3`, `1.0.66-1`,
`1.0.69-0`, `1.0.69-1`, `1.0.69-2`), while healthy long-running processes checked at the
same time used un-suffixed dirs (`1.0.69`, `1.0.70`) - a 7/7 vs 2/2 correlation.
**4. An ETW CPU-sampling trace** (`wpr.exe -start CPU -filemode`, ~8s, analyzed with
`xperf -a profile -detail`) targeting one stuck process showed its top CPU consumers as:
| Module | Usage % |
|---|---|
| `ntoskrnl.exe` (kernel) | 3.82 |
| `FLTMGR.SYS` (filter manager) | 0.78 |
| `copilot.exe.old` | 0.73 |
| `Ntfs.sys` | 0.47 |
| `ntdll.dll` | 0.21 |
| `cldflt.sys` (cloud-files/OneDrive filter), `luafv.sys` (UAC virtualization), `bindflt.sys`, `fileinfo.sys`, `Wof.sys` | ~0.01 each |
i.e. the hot samples land inside the renamed image itself, dominated by kernel and
filesystem-filter-driver time - consistent with a tight retry/poll loop making constant
filesystem syscalls (e.g. against the renamed binary, the pkg cache, or a file-watch path),
not a genuine idle wait.
**5. Not tab/window related:** the parent shells (`pwsh.exe`/`powershell.exe`) hosting these
sessions were all still alive and responding, i.e. this isn't caused by a closed/detached
terminal tab - the CLI process itself is what's stuck.
### Steps to reproduce
1. Start a long-running interactive or `--plan` `copilot` session on Windows and leave it
idle (no active turn) in a Windows Terminal tab.
2. Let an in-place auto-update occur while the session is still running (this can take
days of uptime to hit naturally, or may be reproducible faster by forcing an update
cycle while a session is open).
3. Check whether the still-running process's live image name has become `copilot.exe.old`
(via `QueryFullProcessImageName` / PowerShell 7's `(Get-Process -Id ).ProcessName`)
and whether its CPU usage has pegged one core continuously (`Get-Process -Id |
Select Threads` shows one thread stuck in `Running` with a fast-growing
`TotalProcessorTime`).
### Expected behavior
Either the CLI detects that its own binary/runtime cache has been replaced and gracefully
exits or reloads, or - at minimum - it should never spin a thread at 100% while otherwise
idle. Idle CPU should be ~0% regardless of whether an update occurred underneath a
long-running session.
### Workaround
I wrote a small, self-contained PowerShell script that detects and optionally kills these
stale/spinning processes (checks the live per-PID image name via `QueryFullProcessImageName`
so it works correctly under both Windows PowerShell 5.1 and PowerShell 7, then optionally
re-samples CPU to confirm sustained spin before offering an interactive kill menu). Happy to
share it here or as a gist if useful for triage/repro.
### Possibly related
- #2702 - `launch_engine ACCESS_VIOLATION (0xC0000005) on Windows after auto-update
replaces binary` - same `copilot.exe`/`copilot.exe.old` rename-in-place mechanism, but a
different symptom (crash on next launch vs. this issue's continuous spin on the
already-running old process).
- #3700 / #2208 - WSL2 CLI-spins-at-high-CPU-while-idle reports - same general "busy loop
instead of idle" symptom family, but on a different platform (WSL2, not Windows native)
and not tied to an update/rename event as far as I can tell from those reports, so likely
a distinct root cause.
贡献指南
调研方向
该 payload 未指定源文件或测试;首先在就地自动更新期间复现一个空闲的 Windows 交互式会话或 --plan 会话,然后使用 QueryFullProcessImageName 检查 self-update 路径和 copilot.exe 进程映像。完成的标准是被替换的进程能够安全退出或重新加载,并且空闲会话的 CPU 占用率保持在接近 0% 的水平,而不是无限循环。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- powershell, shell
- 领域
- cli, operating-systems, performance
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 活跃
- 描述清晰度
- 基本清楚
- 新手友好度
- 52/100