github / github/copilot-sdk

Responses API should support native `previous_response_id` chaining

Đang mở
#2,184 3 bình luận 0 reaction 0 người được giao Xem trên GitHub
enhancement
Ngôn ngữ chính
Java
Star
10.5k
Fork
1.5k
Merge trung bình
1 ngày 11 giờ
Pull request đã merge (30 ngày)
128

Mô tả

### Summary

When using the Copilot SDK/CLI with the Responses API, every model round reconstructs and sends the complete conversation history. The runtime does not send `previous_response_id` and sets `store: false`.

This substantially increases request tokens for long-running and resumed sessions.

### Versions tested

- `GitHub.Copilot.SDK` 1.0.5 with Copilot CLI 1.0.73
- `GitHub.Copilot.SDK` 1.0.9-preview.1 with Copilot CLI 1.0.74
- Azure AI Foundry OpenAI-compatible provider
- `wireApi: responses`

The preview SDK references CLI `1.0.76-5`, but that package was not available for testing.

### Observed behavior

A privacy-safe request probe showed:

- Initial request: 1 input item
- Tool continuation: 3 input items
- Next user turn: 5 input items
- Resumed session in a new process: 7 input items

No request included `previous_response_id`. The runtime also emitted `store: false`.

In a real long-running session, this resulted in approximately 26K–28K reported input tokens per request, including tool continuations.

### Expected behavior

After the initial Responses request:

1. Set `store: true`, or otherwise use provider-supported response retention.
2. Capture the terminal response ID.
3. Send subsequent model rounds with:
- `previous_response_id`
- only newly added input items
4. Preserve instructions, tools, and request settings as required by the Responses API.
5. Fall back to full history if the provider rejects or cannot resolve the response ID.

This should apply to both tool-result continuations and subsequent user turns.

### Workaround validation

We implemented an experimental request handler that:

- changes `store: false` to `store: true`,
- captures IDs from terminal SSE response events,
- adds `previous_response_id`,
- replaces reconstructed history with incremental input,
- falls back to full history when chaining cannot be verified.

Live validation reduced:

- a tool continuation from 60 input items to 1,
- a subsequent user turn from 66 input items to 1.

Conversation context and tool behavior remained correct.

The handler also had to account for non-semantic differences when the CLI reconstructed response items:

- regenerated top-level response item `id`,
- omitted assistant-message `phase`,
- omitted output-text `logprobs`.

Semantic content and tool `call_id` values were still compared exactly.

### Request

Could native Responses API chaining be added to the Copilot CLI/runtime and exposed or enabled through the SDK?

It would also be useful to clarify:

- whether the SDK’s `PreviousResponseId` protocol field is currently used by any provider path,
- how response IDs should persist across SDK session resumes or process restarts,
- whether token telemetry can report the actual rewritten wire input rather than reconstructed local history.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start by tracing the Responses API request path and the SDK's PreviousResponseId protocol field, then compare tool continuations, later user turns, and resumed sessions. Done means supported providers use retained response IDs with incremental input, preserve required settings, fall back to full history when chaining fails, and keep context and tool behavior correct.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Lĩnh vực
api, backend-api-design, cli
Loại issue
Tính năng
Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức độ hoạt động
Sôi nổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
35/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.