github / github/copilot-cli

[BUG] Skills prompt injection silently truncates most skills with no prioritization

Đang mở
#2,314 3 bình luận 3 reaction 0 người được giao Xem trên GitHub
area:networking
Ngôn ngữ chính
Shell
Star
11.2k
Fork
1.9k
Merge trung bình
14 giờ 16 phút
Pull request đã merge (30 ngày)
6

Mô tả

### Problem

The SDK truncates the skills list injected into the system prompt using a fixed character budget (`Gyt()` in minified source). When a user has more skills than fit in the budget, the remaining skills are silently dropped with only an HTML comment noting the truncation:

```

```

This means:
- **176 of 201 installed skills are invisible** to the model
- The `skill` tool **fails at runtime** even when the user explicitly names a truncated skill by name
- The user receives **no notification** that skills were dropped

### Root Cause

`Q3r` in `@github/copilot/sdk/index.js` (~line 1362):

```javascript
function Q3r(t) {
let budget = Gyt(); // fixed character budget
let rendered = [], charCount = 0;
for (let skill of t) {
let xml = w3r(skill);
if (charCount + xml.length + 1 > budget) break; // hard stop
charCount += xml.length + 1;
rendered.push(xml);
}
// truncation comment appended if skills were dropped
}
```

Issues with this approach:
1. **No prioritization** — skills are iterated in arrival order (appears alphabetical). No ranking by usage frequency, recency, or relevance to the current conversation.
2. **Character-based budget, not token-based** — `xml.length` is a rough proxy.
3. **Not model-aware** — same budget whether the model has 200K or 1M context. On a 1M context model, all 201 skills would consume ~1.6% of available context.
4. **Silent failure** — the `skill` tool still appears available but invocations fail for truncated skills with a confusing "Skill not found" error.

### Token Math

| | Chars | Est. Tokens |
|---|---|---|
| Current 25 skills (visible) | ~13,000 | ~3,300 |
| All 201 skills | ~65,000 | ~16,400 |
| Hidden skills | ~52,000 | ~13,000 |

- **% of 200K context:** 8.2%
- **% of 1M context:** 1.6%

### Expected Behavior

- All installed skills should be discoverable by the model, or at minimum prioritized intelligently
- The `skill` tool should be able to invoke any installed skill regardless of prompt truncation
- Users should be notified if skills are being dropped

### Suggested Improvements

- **Decouple skill routing from prompt injection** — the `skill` tool should resolve skill names against the full installed list, not just what's in the prompt
- **Model-aware budgets** — scale the budget based on the model's context window
- **Prioritization** — rank by usage frequency, recency, or semantic relevance
- **Lazy loading** — inject only skill names/triggers compactly, load full definitions on match
- **User notification** — surface which skills are hidden, e.g. in `copilot skills list`

### Environment

- Copilot CLI version: 0.0.412
- OS: Windows 11
- Model: Claude Opus 4.6 (1M context)
- Skills installed: 201

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start in @github/copilot/sdk/index.js around Q3r and Gyt, then trace how the rendered skills list connects to skill-tool resolution. Reproduce the truncation with 201 installed skills and verify which names are omitted and how invocation fails. Done means the chosen routing, budgeting, prioritization, and user-notification behavior is implemented and truncated skills no longer fail silently.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
javascript
Lĩnh vực
ai, backend-api-design
Loại issue
Lỗi
Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
35/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.