github / github/app

GPT models cannot follow skills that ask multiple questions in rounds

未关闭
#3,110 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
没有语言数据
星标
2.1k
派生
153
PR 合并指标
30 天内没有已合并 PR

描述

## What happened?

I use Matt Pocock’s [`grill-me` skill](https://github.com/mattpocock/skills), specifically the round-based workflow introduced in v1.2 and tested with v1.2.3.

The skill builds a design tree and asks the entire current “frontier” of independent decisions in one Markdown response:

```text
❓ Q1 — ...
➡️ Recommended answer

❓ Q2 — ...
➡️ Recommended answer
```

This is much more efficient than asking every independent question in a separate turn.

With GPT-5.6 Sol, GitHub Copilot App refuses this format because questions must use the single-question `ask_user` UI. The skill is therefore forced into sequential questioning and cannot follow its documented protocol.

The same skill works with Claude Sonnet 5 in GitHub Copilot App. Claude presents the complete frontier directly in chat and waits for one combined response.

## Steps to reproduce

1. Install Matt Pocock’s skills v1.2 or later.
2. Select GPT-5.6 Sol.
3. Invoke `/grill-me` for a design containing several independent decisions.
4. Ask it to present the current frontier as one round.
5. Observe that it uses, or insists on using, the single-question UI.
6. Repeat with Claude Sonnet 5.
7. Observe that Sonnet renders the complete question round directly in chat.

## Expected behavior

Skills should be able to define a multi-question round protocol consistently across models.

Possible solutions:

- Permit numbered questions in normal chat when explicitly required by an invoked skill.
- Extend `ask_user` to support several independently answerable questions in one round.

## Actual behavior

GPT models follow a Copilot App instruction requiring every question to use the single-question UI. This overrides the skill’s explicit round-based protocol.

Claude Sonnet 5 follows the same skill as written and presents multiple numbered questions in one chat response.

## Impact

Sequential UI prompts add unnecessary turns and undermine the primary efficiency improvement introduced by `grill-me` v1.2.

Behavior also changes depending on the selected model, although the same skill and Copilot App environment are used.

## Environment

- GitHub Copilot App 1.0.80
- Matt Pocock skills v1.2.3
- GPT-5.6 Sol: round-based workflow blocked by single-question policy
- Claude Sonnet 5: round-based workflow works

贡献指南

打开贡献指南

调研方向

首先,在 GitHub Copilot App 1.0.80 中使用 GPT-5.6 Sol 和 Claude Sonnet 5,并借助 Matt Pocock 的 skills v1.2.3,复现基于轮次的 /grill-me 工作流。完成标准是:多问题轮次协议能够在不同模型之间稳定运行,可以通过普通聊天或多问题 ask_user UI 实现。

由索引模型根据 Issue 内容生成。

评估

技术栈
github
领域
ai, desktop
Issue 类型
缺陷
难度
5/5
预计耗时
一周以上
活跃度
活跃
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。