github / github/copilot-cli

Idle Copilot CLI enters `FileWatch` event storm, consumes two CPU cores, and writes a 33+ GB log

オープン
#4,807 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

triage
主要言語
Shell
スター
11.2k
フォーク
1.9k
平均マージ
14時間 16分
マージ済み PR(30日)
6

説明

Describe the bug

Summary

An idle Copilot CLI process launched by Agency entered a sustained FileWatch event storm. The process consumed approximately 221% CPU for more than 35 hours and continuously appended rejected file-watch events to its debug log.

The affected log was 33 GB when first investigated and later exceeded 37 GB. Measured log growth was approximately 216 KB/s, with total process writes around 292 KB/s.

Environment

  • Linux x86_64 under WSL2
  • Kernel: 5.15.167.4-microsoft-standard-WSL2
  • Copilot CLI: 1.0.82
  • Binary build ID: 2b4b8482cccada194dc991f2b088278437f71d7e
  • Process uptime when initially observed: approximately 35 hours
  • Process threads: 111
  • Agency session was open but idle

Actual behavior

  • Sustained process CPU: approximately 221%
  • Resident memory: approximately 1.4 GiB during initial profiling
  • Cumulative CPU time: approximately 77.6 hours after approximately 35 hours of process uptime
  • The last 10,000 log lines spanned approximately 7.1 seconds.
  • Of those lines, 9,982 were:
[DEBUG] [rust:copilot_runtime::protocol::jsonrpc::engine] No connection accepted a host event {"kind":"FileWatch"}

This is approximately 1,400 rejected FileWatch events per second.

No files under the session workspace had modification timestamps in the preceding two minutes, and the process had no active network sockets. This does not appear to be legitimate filesystem activity or active model work.

Other idle Copilot CLI 1.0.83 sessions on the same host consumed approximately 3-4% CPU each. The storm was isolated to the older 1.0.82 process.

Expected behavior

An idle session should not continuously consume multiple CPU cores or produce an unbounded stream of rejected file-watch events. Repeated equivalent events should be coalesced, rate-limited, or dropped without per-event debug logging.

Performance profile collection

Several complementary profiles were collected while the issue was continuously reproducible:

  1. Per-thread CPU and process-accounting snapshot
  2. Three repeated native stack snapshots using eu-stack
  3. Ten-second syscall profile using strace -f -c
  4. Thirty-second Samply attachment attempt at 99 Hz

Samply attached successfully after temporarily relaxing the Linux perf limits, but WSL returned zero samples. The Samply file is retained as evidence of that profiler limitation, but the actionable data is in the thread accounting, eu-stack, and strace captures.

Hot-thread accounting
TID State Instant CPU Cumulative CPU
832 R 54.5% unavailable
849 S 27.3% 101 minutes
853 R 18.2% 161 minutes
855 R 27.3% 123 minutes
858 S 18.2% 240 minutes

The work was distributed across the main thread and native runtime workers rather than being a transient single-thread spike.

Top native stacks

The main thread repeatedly appeared in this Node/V8 foreground-task path:

__memcpy_avx_unaligned_erms
...
v8::platform::DefaultJobState::Join()
v8::platform::DefaultJobHandle::Join()
...
node::PerIsolatePlatformData::FlushForegroundTasksInternal()
...
uv_run
node::SpinEventLoopInternal(node::Environment*)
node::NodeMainInstance::Run()
node::Start(int, char**)

Hot worker TIDs repeatedly showed this shape:

syscall
<frames inside prebuilds/linux-x64/runtime.node>
...
start_thread
__clone3

One repeated worker snapshot was actively writing:

write
<frames inside prebuilds/linux-x64/runtime.node>
...
start_thread
__clone3

The runtime.node binary is stripped, so internal Rust frames could not be symbolicated locally. The repeated main-thread V8 job join/task flush, hot native runtime workers, and continuous copilot_runtime::protocol::jsonrpc::engine logging strongly correlate the CPU consumption with runtime event dispatch and rejection rather than filesystem scanning alone.

Syscall profile

The ten-second strace -f -c capture across 111 threads recorded 88,712 calls:

Syscall Calls Errors
futex 52,893 6,863
write 10,646 0
read 9,648 1,420
epoll_pwait 6,225 0
epoll_wait 3,626 0
statx 2,718 0
openat 1,359 0
close 1,359 0

The high futex rate shows heavy worker synchronization, while more than 1,000 writes per second matches the observed per-event debug-log storm. The repeated openat/statx calls also indicate ongoing filesystem-related processing despite the workspace being idle.

Likely culprit

The strongest evidence points to a feedback loop or queued-event replay in the native Copilot runtime:

  1. FileWatch host events are generated at approximately 1,400 events/s.
  2. No connection accepts the events.
  3. Each rejection is logged individually.
  4. Native runtime.node workers remain hot and heavily synchronized.
  5. The Node main thread repeatedly joins V8 jobs and flushes foreground tasks.
  6. No corresponding workspace changes or network activity explain the event rate.

Potential areas to investigate:

  • File-watch event production and deduplication
  • Host-event dispatch when no connection accepts an event
  • Requeue/retry behavior for rejected host events
  • Per-event debug logging without rate limiting
  • Whether the log write or another watched path can feed new events back into the watcher

Reproduction information

The issue was observed in an Agency-launched Copilot CLI 1.0.82 session left open and idle for approximately 35 hours. A deterministic trigger is not yet known. Once present, the behavior persisted indefinitely and remained reproducible throughout profiling.

Affected version

No response

Steps to reproduce the behavior

No response

Expected behavior

No response

Additional context

No response

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

調査の方向性

まず、copilot_runtime::protocol::jsonrpc::engine の logging path と、レポートで説明されている file-watch イベントの生成、ディスパッチ、リトライの領域から着手してください。最初に eu-stack と strace の調査結果を使用してください。Samply のプロファイルには利用可能なサンプルがありません。アイドル状態のセッションで、FileWatch の拒否イベントが継続的に大量発生する状態、マルチコア CPU 使用率の上昇、イベントごとの無制限な logging が発生しなくなれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
node.js, rust
領域
cli, performance
issue の種類
バグ
難易度
5/5
見積もり時間
1週間以上
活発さ
活発
明瞭さ
説明が足りない
初心者へのやさしさ
25/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。