github / github/app

GPT models cannot follow skills that ask multiple questions in rounds

Đang mở
#3,110 2 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Không có dữ liệu ngôn ngữ
Star
2.1k
Fork
153
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

## What happened?

I use Matt Pocock’s [`grill-me` skill](https://github.com/mattpocock/skills), specifically the round-based workflow introduced in v1.2 and tested with v1.2.3.

The skill builds a design tree and asks the entire current “frontier” of independent decisions in one Markdown response:

```text
❓ Q1 — ...
➡️ Recommended answer

❓ Q2 — ...
➡️ Recommended answer
```

This is much more efficient than asking every independent question in a separate turn.

With GPT-5.6 Sol, GitHub Copilot App refuses this format because questions must use the single-question `ask_user` UI. The skill is therefore forced into sequential questioning and cannot follow its documented protocol.

The same skill works with Claude Sonnet 5 in GitHub Copilot App. Claude presents the complete frontier directly in chat and waits for one combined response.

## Steps to reproduce

1. Install Matt Pocock’s skills v1.2 or later.
2. Select GPT-5.6 Sol.
3. Invoke `/grill-me` for a design containing several independent decisions.
4. Ask it to present the current frontier as one round.
5. Observe that it uses, or insists on using, the single-question UI.
6. Repeat with Claude Sonnet 5.
7. Observe that Sonnet renders the complete question round directly in chat.

## Expected behavior

Skills should be able to define a multi-question round protocol consistently across models.

Possible solutions:

- Permit numbered questions in normal chat when explicitly required by an invoked skill.
- Extend `ask_user` to support several independently answerable questions in one round.

## Actual behavior

GPT models follow a Copilot App instruction requiring every question to use the single-question UI. This overrides the skill’s explicit round-based protocol.

Claude Sonnet 5 follows the same skill as written and presents multiple numbered questions in one chat response.

## Impact

Sequential UI prompts add unnecessary turns and undermine the primary efficiency improvement introduced by `grill-me` v1.2.

Behavior also changes depending on the selected model, although the same skill and Copilot App environment are used.

## Environment

- GitHub Copilot App 1.0.80
- Matt Pocock skills v1.2.3
- GPT-5.6 Sol: round-based workflow blocked by single-question policy
- Claude Sonnet 5: round-based workflow works

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start by reproducing the round-based /grill-me workflow in GitHub Copilot App 1.0.80 with GPT-5.6 Sol and Claude Sonnet 5, using Matt Pocock’s skills v1.2.3. Done means the multi-question round protocol works consistently across models, either through normal chat or a multi-question ask_user UI.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
github
Lĩnh vực
ai, desktop
Loại issue
Lỗi
Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức độ hoạt động
Sôi nổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
35/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.