Active Agent Host sessions can retain stale synchronized skill paths
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
## Type
Bug
## Summary
A synchronized skill can remain advertised and resolvable by name in an active Agent Host session while its versioned plugin cache directory has already been replaced. Invoking the skill then fails with `ENOENT`, even though a newer cache version contains the same skill.
This is an independent reproduction of the general problem reported in #328464, using `create-draft-pr` rather than `act-on-feedback`. This reproduction includes the verbatim session event records and shows the affected cache version changing while the session remains active.
## Steps to reproduce
1. Start an Agent Host session in Code Insiders with synchronized skills enabled.
2. Keep the session active while synchronized customization plugin cache versions are refreshed.
3. Ask the agent to create a draft PR.
4. The agent invokes the advertised `create-draft-pr` skill.
5. Observe that loading `SKILL.md` fails because the path retained by the active session points to an evicted version directory.
## Actual behavior
The skill was advertised by the Copilot CLI customization provider. Code Insiders logged:
```text
2026-08-15 09:08:03.534 [debug] [CopilotCLICustomizationProvider] skills (44): grill-me, improve-codebase-architecture, playwright-cli, svg-diagram, agent-host-e2e-coverage-round, code-oss-dev, create-pr-fast, daily-telemetry-dashboard, github-notification-cleanup, vscode-scenario-builder, vscode-telemetry-direct, dap-cli, inbox-add-reaction, inbox-dismiss-notification, inbox-get-notification-details, inbox-install-gh-cli, inbox-list-notifications, inbox-manage-subscription, inbox-mark-all-read, inbox-memory, inbox-open-notification, inbox-search-issues, inbox-setup, inbox-storage, monitor-pr, context, finalize, help, interface-planner, reconcile, troubleshoot, agent-customization, act-on-feedback, code-review, commit, create-draft-pr, create-pr, fix-ci, generate-run-commands, merge, sync, sync-upstream, update-pr, update-skills
```
The active session's `events.jsonl` then recorded the invocation:
```json
{"type":"tool.execution_start","data":{"toolCallId":"call_5yYCB4lr3KYxH2ggOFqOqOTP","toolName":"skill","arguments":{"skill":"create-draft-pr"},"turnId":"0","model":"gpt-5.6-sol","rte":true},"id":"d30bdbb7-e83a-43a8-a4d3-01657398bff0","timestamp":"2026-08-15T18:08:20.441Z","parentId":"b13d3123-b2cc-4705-bc02-a482723f5cd3"}
```
The post-tool-use failure was:
```json
{"type":"hook.start","data":{"hookInvocationId":"a059c72e-3f4c-4835-a27a-77c046537eee","hookType":"postToolUseFailure","input":{"sessionId":"6b44c1e2-0044-4c78-b6c3-f76aebd3fe41","timestamp":1786817300541,"cwd":"/Users/roblou/code/vscode.worktrees/vscode-terminal-hang-debugging","toolName":"skill","toolArgs":"{\"skill\":\"create-draft-pr\"}","error":"Failed to read skill file: Error: ENOENT: no such file or directory, open '/Users/roblou/Library/Application Support/Code - Insiders/agentPlugins/vscode-synced-customization-agent-host-copilotcli-671424081/1943847843/skills/create-draft-pr/SKILL.md'"}},"id":"9f21a4e9-e516-4c5f-b845-985c514bc62e","timestamp":"2026-08-15T18:08:20.541Z","parentId":"ed530e79-33b1-490f-a885-effedd3ebc66"}
```
The corresponding completion event confirms the skill was found in discovery but could not be read:
```json
{"type":"tool.execution_complete","data":{"toolCallId":"call_5yYCB4lr3KYxH2ggOFqOqOTP","model":"gpt-5.6-sol","interactionId":"d08b02da-4490-4d00-88f2-0298b0fd130a","turnId":"0","rte":true,"success":false,"error":{"message":"Failed to read skill file: Error: ENOENT: no such file or directory, open '/Users/roblou/Library/Application Support/Code - Insiders/agentPlugins/vscode-synced-customization-agent-host-copilotcli-671424081/1943847843/skills/create-draft-pr/SKILL.md'","code":"failure"},"toolTelemetry":{"properties":{"skillNameHash":"e3b4a2ba6bc554b0826cf7e5b0d3daa2f0c8da59b9aa699e823630612710ed49","skillSource":"plugin","found":"true","readError":"true"},"restrictedProperties":{"skillName":"create-draft-pr","errorMessage":"Error: ENOENT: no such file or directory, open '/Users/roblou/Library/Application Support/Code - Insiders/agentPlugins/vscode-synced-customization-agent-host-copilotcli-671424081/1943847843/skills/create-draft-pr/SKILL.md'"}}},"id":"a6ba960d-dfb4-4617-868e-9ddbf0ff8414","timestamp":"2026-08-15T18:08:20.544Z","parentId":"abc7ce53-c182-4d1d-a252-bc0cf162a5b7"}
```
## Cache version evidence
At failure time, the retained path referenced version directory:
```text
vscode-synced-customization-agent-host-copilotcli-671424081/1943847843/skills/create-draft-pr/SKILL.md
```
That directory no longer existed. During investigation, the same plugin root successively contained `26906424` and then `1628134047`. The current version contains:
```text
vscode-synced-customization-agent-host-copilotcli-671424081/1628134047/skills/create-draft-pr/SKILL.md
```
This indicates that discovery metadata held by the active session outlived the versioned cache directory it referenced.
## Expected behavior
- Paths exposed to an active session must remain readable for that session's lifetime, or be resolved indirectly through a stable plugin location.
- Cache refresh/eviction should atomically update active customization metadata.
- On `ENOENT`, the skill loader should refresh and retry resolution against the current synchronized plugin version.
- A skill must not report `found: true` while its backing file is unreadable.
## Suggested areas to investigate
- Lifetime of versioned synchronized plugin cache directories relative to active Agent Host sessions.
- Whether skill discovery stores absolute versioned paths instead of stable identifiers.
- Atomicity of customization publication and old-version eviction.
- Re-resolution/retry behavior after `ENOENT`.
(Written by Copilot)
Contributor guide
Assessment
This issue has not been assessed yet.