Idle Copilot CLI enters `FileWatch` event storm, consumes two CPU cores, and writes a 33+ GB log
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 11.2k
- Forks
- 1.9k
- Avg merge
- 14h 16m
- Merged PRs (30d)
- 6
Description
Describe the bug
Summary
An idle Copilot CLI process launched by Agency entered a sustained FileWatch event storm. The process consumed approximately 221% CPU for more than 35 hours and continuously appended rejected file-watch events to its debug log.
The affected log was 33 GB when first investigated and later exceeded 37 GB. Measured log growth was approximately 216 KB/s, with total process writes around 292 KB/s.
Environment
- Linux x86_64 under WSL2
- Kernel:
5.15.167.4-microsoft-standard-WSL2 - Copilot CLI:
1.0.82 - Binary build ID:
2b4b8482cccada194dc991f2b088278437f71d7e - Process uptime when initially observed: approximately 35 hours
- Process threads: 111
- Agency session was open but idle
Actual behavior
- Sustained process CPU: approximately 221%
- Resident memory: approximately 1.4 GiB during initial profiling
- Cumulative CPU time: approximately 77.6 hours after approximately 35 hours of process uptime
- The last 10,000 log lines spanned approximately 7.1 seconds.
- Of those lines, 9,982 were:
[DEBUG] [rust:copilot_runtime::protocol::jsonrpc::engine] No connection accepted a host event {"kind":"FileWatch"}
This is approximately 1,400 rejected FileWatch events per second.
No files under the session workspace had modification timestamps in the preceding two minutes, and the process had no active network sockets. This does not appear to be legitimate filesystem activity or active model work.
Other idle Copilot CLI 1.0.83 sessions on the same host consumed approximately 3-4% CPU each. The storm was isolated to the older 1.0.82 process.
Expected behavior
An idle session should not continuously consume multiple CPU cores or produce an unbounded stream of rejected file-watch events. Repeated equivalent events should be coalesced, rate-limited, or dropped without per-event debug logging.
Performance profile collection
Several complementary profiles were collected while the issue was continuously reproducible:
- Per-thread CPU and process-accounting snapshot
- Three repeated native stack snapshots using
eu-stack - Ten-second syscall profile using
strace -f -c - Thirty-second Samply attachment attempt at 99 Hz
Samply attached successfully after temporarily relaxing the Linux perf limits, but WSL returned zero samples. The Samply file is retained as evidence of that profiler limitation, but the actionable data is in the thread accounting, eu-stack, and strace captures.
Hot-thread accounting
| TID | State | Instant CPU | Cumulative CPU |
|---|---|---|---|
| 832 | R | 54.5% | unavailable |
| 849 | S | 27.3% | 101 minutes |
| 853 | R | 18.2% | 161 minutes |
| 855 | R | 27.3% | 123 minutes |
| 858 | S | 18.2% | 240 minutes |
The work was distributed across the main thread and native runtime workers rather than being a transient single-thread spike.
Top native stacks
The main thread repeatedly appeared in this Node/V8 foreground-task path:
__memcpy_avx_unaligned_erms
...
v8::platform::DefaultJobState::Join()
v8::platform::DefaultJobHandle::Join()
...
node::PerIsolatePlatformData::FlushForegroundTasksInternal()
...
uv_run
node::SpinEventLoopInternal(node::Environment*)
node::NodeMainInstance::Run()
node::Start(int, char**)
Hot worker TIDs repeatedly showed this shape:
syscall
<frames inside prebuilds/linux-x64/runtime.node>
...
start_thread
__clone3
One repeated worker snapshot was actively writing:
write
<frames inside prebuilds/linux-x64/runtime.node>
...
start_thread
__clone3
The runtime.node binary is stripped, so internal Rust frames could not be symbolicated locally. The repeated main-thread V8 job join/task flush, hot native runtime workers, and continuous copilot_runtime::protocol::jsonrpc::engine logging strongly correlate the CPU consumption with runtime event dispatch and rejection rather than filesystem scanning alone.
Syscall profile
The ten-second strace -f -c capture across 111 threads recorded 88,712 calls:
| Syscall | Calls | Errors |
|---|---|---|
futex |
52,893 | 6,863 |
write |
10,646 | 0 |
read |
9,648 | 1,420 |
epoll_pwait |
6,225 | 0 |
epoll_wait |
3,626 | 0 |
statx |
2,718 | 0 |
openat |
1,359 | 0 |
close |
1,359 | 0 |
The high futex rate shows heavy worker synchronization, while more than 1,000 writes per second matches the observed per-event debug-log storm. The repeated openat/statx calls also indicate ongoing filesystem-related processing despite the workspace being idle.
Likely culprit
The strongest evidence points to a feedback loop or queued-event replay in the native Copilot runtime:
FileWatchhost events are generated at approximately 1,400 events/s.- No connection accepts the events.
- Each rejection is logged individually.
- Native
runtime.nodeworkers remain hot and heavily synchronized. - The Node main thread repeatedly joins V8 jobs and flushes foreground tasks.
- No corresponding workspace changes or network activity explain the event rate.
Potential areas to investigate:
- File-watch event production and deduplication
- Host-event dispatch when no connection accepts an event
- Requeue/retry behavior for rejected host events
- Per-event debug logging without rate limiting
- Whether the log write or another watched path can feed new events back into the watcher
Reproduction information
The issue was observed in an Agency-launched Copilot CLI 1.0.82 session left open and idle for approximately 35 hours. A deterministic trigger is not yet known. Once present, the behavior persisted indefinitely and remained reproducible throughout profiling.
Affected version
No response
Steps to reproduce the behavior
No response
Expected behavior
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the copilot_runtime::protocol::jsonrpc::engine logging path and the file-watch event production, dispatch, and retry areas described in the report. Use the eu-stack and strace findings first; the Samply profile has no usable samples. Done means an idle session no longer produces a sustained rejected FileWatch storm, multi-core CPU usage, or unbounded per-event logging.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, rust
- Domain
- cli, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100