github / github/copilot-cli

[BUG] Skills prompt injection silently truncates most skills with no prioritization

Aberta
#2,314 3 comentários 3 reações 0 responsáveis Ver no GitHub
area:networking
Linguagem predominante
Shell
Estrelas
11.2k
Forks
1.9k
Merge médio
14h 16min
PRs com merge (30d)
6

Descrição

### Problem

The SDK truncates the skills list injected into the system prompt using a fixed character budget (`Gyt()` in minified source). When a user has more skills than fit in the budget, the remaining skills are silently dropped with only an HTML comment noting the truncation:

```

```

This means:
- **176 of 201 installed skills are invisible** to the model
- The `skill` tool **fails at runtime** even when the user explicitly names a truncated skill by name
- The user receives **no notification** that skills were dropped

### Root Cause

`Q3r` in `@github/copilot/sdk/index.js` (~line 1362):

```javascript
function Q3r(t) {
let budget = Gyt(); // fixed character budget
let rendered = [], charCount = 0;
for (let skill of t) {
let xml = w3r(skill);
if (charCount + xml.length + 1 > budget) break; // hard stop
charCount += xml.length + 1;
rendered.push(xml);
}
// truncation comment appended if skills were dropped
}
```

Issues with this approach:
1. **No prioritization** — skills are iterated in arrival order (appears alphabetical). No ranking by usage frequency, recency, or relevance to the current conversation.
2. **Character-based budget, not token-based** — `xml.length` is a rough proxy.
3. **Not model-aware** — same budget whether the model has 200K or 1M context. On a 1M context model, all 201 skills would consume ~1.6% of available context.
4. **Silent failure** — the `skill` tool still appears available but invocations fail for truncated skills with a confusing "Skill not found" error.

### Token Math

| | Chars | Est. Tokens |
|---|---|---|
| Current 25 skills (visible) | ~13,000 | ~3,300 |
| All 201 skills | ~65,000 | ~16,400 |
| Hidden skills | ~52,000 | ~13,000 |

- **% of 200K context:** 8.2%
- **% of 1M context:** 1.6%

### Expected Behavior

- All installed skills should be discoverable by the model, or at minimum prioritized intelligently
- The `skill` tool should be able to invoke any installed skill regardless of prompt truncation
- Users should be notified if skills are being dropped

### Suggested Improvements

- **Decouple skill routing from prompt injection** — the `skill` tool should resolve skill names against the full installed list, not just what's in the prompt
- **Model-aware budgets** — scale the budget based on the model's context window
- **Prioritization** — rank by usage frequency, recency, or semantic relevance
- **Lazy loading** — inject only skill names/triggers compactly, load full definitions on match
- **User notification** — surface which skills are hidden, e.g. in `copilot skills list`

### Environment

- Copilot CLI version: 0.0.412
- OS: Windows 11
- Model: Claude Opus 4.6 (1M context)
- Skills installed: 201

Guia de contribuição

Abrir o guia de contribuição

Direção de pesquisa

Start in @github/copilot/sdk/index.js around Q3r and Gyt, then trace how the rendered skills list connects to skill-tool resolution. Reproduce the truncation with 201 installed skills and verify which names are omitted and how invocation fails. Done means the chosen routing, budgeting, prioritization, and user-notification behavior is implemented and truncated skills no longer fail silently.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
javascript
Domínio
ai, backend-api-design
Tipo de issue
Bug
Dificuldade
5/5
Tempo estimado
Mais de uma semana
Status de atividade
Pouca atividade
Clareza
Razoavelmente clara
Facilidade para iniciantes
35/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.