GPT models cannot follow skills that ask multiple questions in rounds
- Lingua principale
- Nessun dato sulla lingua
- Stelle
- 2.1k
- Fork
- 153
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
## What happened?
I use Matt Pocock’s [`grill-me` skill](https://github.com/mattpocock/skills), specifically the round-based workflow introduced in v1.2 and tested with v1.2.3.
The skill builds a design tree and asks the entire current “frontier” of independent decisions in one Markdown response:
```text
❓ Q1 — ...
➡️ Recommended answer
❓ Q2 — ...
➡️ Recommended answer
```
This is much more efficient than asking every independent question in a separate turn.
With GPT-5.6 Sol, GitHub Copilot App refuses this format because questions must use the single-question `ask_user` UI. The skill is therefore forced into sequential questioning and cannot follow its documented protocol.
The same skill works with Claude Sonnet 5 in GitHub Copilot App. Claude presents the complete frontier directly in chat and waits for one combined response.
## Steps to reproduce
1. Install Matt Pocock’s skills v1.2 or later.
2. Select GPT-5.6 Sol.
3. Invoke `/grill-me` for a design containing several independent decisions.
4. Ask it to present the current frontier as one round.
5. Observe that it uses, or insists on using, the single-question UI.
6. Repeat with Claude Sonnet 5.
7. Observe that Sonnet renders the complete question round directly in chat.
## Expected behavior
Skills should be able to define a multi-question round protocol consistently across models.
Possible solutions:
- Permit numbered questions in normal chat when explicitly required by an invoked skill.
- Extend `ask_user` to support several independently answerable questions in one round.
## Actual behavior
GPT models follow a Copilot App instruction requiring every question to use the single-question UI. This overrides the skill’s explicit round-based protocol.
Claude Sonnet 5 follows the same skill as written and presents multiple numbered questions in one chat response.
## Impact
Sequential UI prompts add unnecessary turns and undermine the primary efficiency improvement introduced by `grill-me` v1.2.
Behavior also changes depending on the selected model, although the same skill and Copilot App environment are used.
## Environment
- GitHub Copilot App 1.0.80
- Matt Pocock skills v1.2.3
- GPT-5.6 Sol: round-based workflow blocked by single-question policy
- Claude Sonnet 5: round-based workflow works
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start by reproducing the round-based /grill-me workflow in GitHub Copilot App 1.0.80 with GPT-5.6 Sol and Claude Sonnet 5, using Matt Pocock’s skills v1.2.3. Done means the multi-question round protocol works consistently across models, either through normal chat or a multi-question ask_user UI.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- github
- Ambito
- ai, desktop
- Tipo di issue
- Bug
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Attiva
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100