[BUG] Skills prompt injection silently truncates most skills with no prioritization
- Ngôn ngữ chính
- Shell
- Star
- 11.2k
- Fork
- 1.9k
- Merge trung bình
- 14 giờ 16 phút
- Pull request đã merge (30 ngày)
- 6
Mô tả
### Problem
The SDK truncates the skills list injected into the system prompt using a fixed character budget (`Gyt()` in minified source). When a user has more skills than fit in the budget, the remaining skills are silently dropped with only an HTML comment noting the truncation:
```
```
This means:
- **176 of 201 installed skills are invisible** to the model
- The `skill` tool **fails at runtime** even when the user explicitly names a truncated skill by name
- The user receives **no notification** that skills were dropped
### Root Cause
`Q3r` in `@github/copilot/sdk/index.js` (~line 1362):
```javascript
function Q3r(t) {
let budget = Gyt(); // fixed character budget
let rendered = [], charCount = 0;
for (let skill of t) {
let xml = w3r(skill);
if (charCount + xml.length + 1 > budget) break; // hard stop
charCount += xml.length + 1;
rendered.push(xml);
}
// truncation comment appended if skills were dropped
}
```
Issues with this approach:
1. **No prioritization** — skills are iterated in arrival order (appears alphabetical). No ranking by usage frequency, recency, or relevance to the current conversation.
2. **Character-based budget, not token-based** — `xml.length` is a rough proxy.
3. **Not model-aware** — same budget whether the model has 200K or 1M context. On a 1M context model, all 201 skills would consume ~1.6% of available context.
4. **Silent failure** — the `skill` tool still appears available but invocations fail for truncated skills with a confusing "Skill not found" error.
### Token Math
| | Chars | Est. Tokens |
|---|---|---|
| Current 25 skills (visible) | ~13,000 | ~3,300 |
| All 201 skills | ~65,000 | ~16,400 |
| Hidden skills | ~52,000 | ~13,000 |
- **% of 200K context:** 8.2%
- **% of 1M context:** 1.6%
### Expected Behavior
- All installed skills should be discoverable by the model, or at minimum prioritized intelligently
- The `skill` tool should be able to invoke any installed skill regardless of prompt truncation
- Users should be notified if skills are being dropped
### Suggested Improvements
- **Decouple skill routing from prompt injection** — the `skill` tool should resolve skill names against the full installed list, not just what's in the prompt
- **Model-aware budgets** — scale the budget based on the model's context window
- **Prioritization** — rank by usage frequency, recency, or semantic relevance
- **Lazy loading** — inject only skill names/triggers compactly, load full definitions on match
- **User notification** — surface which skills are hidden, e.g. in `copilot skills list`
### Environment
- Copilot CLI version: 0.0.412
- OS: Windows 11
- Model: Claude Opus 4.6 (1M context)
- Skills installed: 201
Hướng dẫn đóng góp
Hướng nghiên cứu
Start in @github/copilot/sdk/index.js around Q3r and Gyt, then trace how the rendered skills list connects to skill-tool resolution. Reproduce the truncation with 201 installed skills and verify which names are omitted and how invocation fails. Done means the chosen routing, budgeting, prioritization, and user-notification behavior is implemented and truncated skills no longer fail silently.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- javascript
- Lĩnh vực
- ai, backend-api-design
- Loại issue
- Lỗi
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 35/100