Github Copilot Tool-result truncation splits UTF-16 surrogate pairs → HTTP 400 `invalid_request_body` on OpenAI-family models (Claude models unaffected)

Type: Bug

## Title
Tool-result truncation splits UTF-16 surrogate pairs → HTTP 400 `invalid_request_body`
on OpenAI-family models (Claude models unaffected)

## Environment
- VS Code: 1.135.0 (08d4889f9ec4a1685d257b9b95de036c8e1ce1e5, x64)
- GitHub Copilot Chat: 0.63.0 (bundled in app)
- OS: Windows 11 Enterprise 10.0.26200 (64-bit)
- Models affected: GPT-5.6 Luna, and other GPT-family models
- Models NOT affected: Claude Opus 4.7 / 4.8 / 5, Claude Sonnet 5
- Workspace: multi-root (4 folders), agent mode

## Summary
In agent mode with a GPT-family model, the request that carries **tool results** back to
the model is rejected by the Copilot proxy:

400 {"error":{"message":"Invalid body: failed to parse JSON value. Please check the value
to ensure it is valid JSON. (Common errors include trailing commas, missing closing
brackets, missing quotation marks, etc.)","code":"invalid_request_body"}}

The identical workspace, prompts and tools work indefinitely on Claude models.

## Impact
The failure is **unrecoverable within the session**. "Try Again" resends the same poisoned
history and fails again, so the conversation must be abandoned. This forces either a new
chat or a switch to a Claude model — and switching mid-thread triggers a *second*
`invalid_request_body` failure mode of its own.

## Evidence
I parsed 20 session transcripts from
`%APPDATA%\Code\User\workspaceStorage\\GitHub.copilot-chat\transcripts\`.
Tool-call ID prefixes identify the model family (`call_*` = GPT, `toolu_*` = Claude).

Tool calls completed before the session died:

| Session | Family | Tool calls |
|----------|--------|-----------:|
| 57c0fb62 | GPT | 2 |
| a23b574f | GPT | 2 |
| 8a97fe50 | GPT | 3 |
| 785f1caa | GPT | 4 |
| 293f1240 | GPT | 5 |
| 6525040d | GPT | 27 |
| d781627a | Claude | 177 |
| 391293c2 | Claude | 634 |

Every GPT session dies on the follow-up request carrying tool results — never on the
first user message. Session 8a97fe50 records two `Try Again` clicks followed by the user
retyping the same question three times.

Ruled out by measurement:
- Context size — GPT-5.6 Luna advertises 922,000 prompt / 128,000 output tokens;
the full customization set is ~49k tokens.
- `mcp.json` — parses cleanly, one server.
- Instruction/skill/agent files — 12 files, 195 KB, all with terminated frontmatter
and zero raw control characters.
- Recorded tool-call `arguments` — validated every payload across all 20 transcripts,
0 invalid.

## Suspected cause
The workspace docs use emoji as status markers, at high density:
- `04-field-level-migration-matrix.md`: 328 astral chars (🟢×158 🔴×105 🟡×40 🔵×25)
in 54,864 chars → **one surrogate pair every ~167 characters**.

`grep_search` truncates long lines (emitting `[match at col 2892 · line truncated,
4,277 chars]`) and large tool results are trimmed. When a cut lands **between the high
and low surrogate of an emoji**, the tool result contains a **lone surrogate**. That value
survives client-side `JSON.stringify` as a `\uD83D`-style escape, and the strict validator
on the OpenAI-family path rejects the body. The Claude path evidently tolerates or
sanitizes it.

**Corroborating artifact:** six characters in those documents are already permanently
corrupted to `U+FFFD` on disk, at exactly the positions where status emoji belong — a
previous agent run truncated mid-emoji and wrote the mangled text back. So the truncation
is demonstrably not surrogate-aware in at least one code path.

## Steps to reproduce
1. Add a markdown file containing emoji roughly every ~150 characters, including at
least one line longer than ~3000 characters with emoji beyond the truncation column.
2. Select a GPT-family model (e.g. GPT-5.6 Luna) in agent mode.
3. Ask the agent a question that triggers a broad `grep_search`
(regex with alternation, `maxResults: 200`, `includePattern: "**"`) over that folder.
4. The tool calls succeed; the next request fails with 400 `invalid_request_body`.
5. "Try Again" fails identically. Repeat with a Claude model — it succeeds.

## Expected
Tool results are sanitized before serialization, so no request is ever emitted that the
endpoint will reject.

## Suggested fix
1. Clamp all truncation to code-point (ideally grapheme-cluster) boundaries rather than
UTF-16 code units or bytes — this affects `grep_search` line truncation, large-result
trimming, and file write-back.
2. Defensively sanitize lone surrogates (replace with `U+FFFD`) when appending tool
results to conversation history.
3. Surface a recoverable client-side error instead of an opaque proxy 400, so the turn
can be retried without discarding the session.

## Related
Switching models mid-conversation produces the same `invalid_request_body` status code
via a different path (replaying one provider's tool/reasoning format into the other's
schema). 3 of the 18 sessions show mixed `call_*` / `toolu_*` IDs. Possibly worth
splitting into a separate issue.

VS Code version: Code 1.135.0 (08d4889f9ec4a1685d257b9b95de036c8e1ce1e5, 2026-08-25T14:26:52Z)
OS version: Windows_NT x64 10.0.26200
Modes:

System Info

|Item|Value|
|---|---|
|CPUs|11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 1498)|
|GPU Status|2d_canvas: enabled
GPU0: VENDOR= 0x8086, DEVICE=0x9a49 [Intel(R) Iris(R) Xe Graphics], DRIVER_VENDOR=Intel, DRIVER_VERSION=32.0.101.7088 *ACTIVE*
GPU1: VENDOR= 0x1414, DEVICE=0x008c [Microsoft Basic Render Driver], DRIVER_VERSION=10.0.26100.8875
Machine model name:
Machine model version:
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
trees_in_viz: disabled_off
video_decode: enabled
video_encode: enabled
webgl: enabled
webgpu: enabled
webnn: disabled_off|
|Load (avg)|undefined|
|Memory (System)|31.73GB (7.78GB free)|
|Process Argv|--crash-reporter-id c1a117bf-2b7f-4d43-b387-9d103e2e994d|
|Screen Reader|no|
|VM|0%|
Extensions (10)

Name|Identifier|Author|Version
---|---|---|---
Markdown Preview Mermaid Support|bierner.markdown-mermaid|bierner|1.32.1
ESLint|dbaeumer.vscode-eslint|dbaeumer|3.0.34
Prettier - Code formatter|esbenp.prettier-vscode|esbenp|12.4.0
Jest / Vitest Runner|firsttris.vscode-jest-runner|firsttris|0.4.148
.NET Install Tool|ms-dotnettools.vscode-dotnet-runtime|ms-dotnettools|3.1.0
PowerShell|ms-vscode.powershell|ms-vscode|2025.4.0
Jest|orta.vscode-jest|Orta|6.4.4
Material Icon Theme|pkief.material-icon-theme|PKief|5.38.1
Live Server|ritwickdey.liveserver|ritwickdey|5.7.10
vscode-icons|vscode-icons-team.vscode-icons|vscode-icons-team|12.19.0

A/B Experiments

```
vsliv368cf:30146710
binariesv615:30325510
nativeloc1:31344060
dwcopilot:31170013
dwoutputs:31242946
copilot_t_ci:31333650
e5gg6876:31282496
pythonrdcb7:31342333
6518g693:31463988
aj953862:31281341
4f60g487:31327383
envsactivate1:31551504
gh487529:31544284
cloudbuttont:31379625
42190218_ostrepl:31403339
ec5jj548:31422691
diffpatch-lysithea-24-production:31579162
cp_cls_t_966_ss:31526232
4je02754:31466945
8hhj4413:31478653
ge8j1254_inline_auto_hint_haiku:31490510
cp_cls_c_1081:31454833
conptydll_true:31498968
e9c30283:31461165
46204921:31447328
ei9d7968:31496641
chat:31457767
8hig5102:31480529
89g7j272:31518289
i2gc6536:31499202
52612955:31516516
ddid_c:31478207
hmra_i5g22:31518061
l_j1ci3728:31575056
7df3h592:31512476
cp_cls_t_1082:31535311
logging_enabled_new:31498466
db5d2638:31499441
jb_cp_cls_t_632:31543129
56dj4588:31512888
32d76977:31512328
ha629193:31508444
a1ije391_t:31540920
jbcp_cls_pctr_t:31531130
cp_intellij_t_nes:31548657
d7b18187:31526828
ahp-both-windows:31556933
ihg5j128:31534457
7g2b5551:31542111
enable_editor_pane_layout:31569726
allow-none:31555437
36h42362:31564511
c7c27ce7:31554789
1h923230:31564177
1532g621_copy:31554320
treatment-23-1:31555779
unuse_dynamic_mcp:31555281
0d8dfbc3:31570291
a1hcc538:31559322
0c1h4866:31566224
vrbsty_fls:31561059
session-mark-done:31558131
5b8j3302:31564601
mangle-name-treatment:31572634
multiv2:31574985

```