[Bug]: Always accidentally failed to parse LLM response into JSON.Error when entering numbers

English Translation

On Android, Midscene can accidentally fail to parse the LLM response into JSON when entering numbers.

The raw response shows that the generated action-param-json contains nested double quotes inside the prompt text (the "1" button on the numeric keypad), which breaks the JSON and causes AIResponseParseError.

The reproduction call is: enter ${amount} and tap Confirm/OK/Done.

Version

Android

Details

Caused by: AIResponseParseError: XML parse error: Failed to parse action-param-json: Error: failed to parse LLM response into JSON. Error - Error: Colon expected at position 42. Response -
{
"locate": {
"prompt": "数字键盘上的 "1" 按钮",
"bbox": [100, 765, 200, 805]
}
}
❯ plan node_modules/@midscene/core/src/ai-model/llm-planning.ts:303:11
❯ Object.executor node_modules/@midscene/core/src/agent/tasks.ts:360:28
❯ TaskRunner.flush node_modules/@midscene/core/src/task-runner.ts:297:25
❯ TaskExecutor.runAction node_modules/@midscene/core/src/agent/tasks.ts:323:22
❯ runAiAct node_modules/@midscene/core/src/agent/agent.ts:840:40
❯ AndroidAgent.aiAct node_modules/@midscene/core/src/agent/agent.ts:878:12
❯ InvestPage.performInputAmount src/pages/android/InvestPage.ts:55:5
❯ src/tests/android/invest_regression.test.ts:29:5

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { usage: { prompt_tokens: 5064, completion_tokens: 198, total_tokens: 5262, cached_input: +0, time_cost: 1718, model_name: 'qwen/qwen3.5-flash-02-23', model_description: 'qwen3.5 mode', intent: 'default', request_id: undefined }, rawResponse: '用户要求输入 "10" 并点击确认。当前页面显示数字键盘，我需要先点击数字 "1"，然后点击 "0"，最后点击 "OK" 按钮。首先点击数字 "1"。\n点击数字键盘上的 "1"\nTap\n\n{\n "locate": {\n "prompt": "数字键盘上的 "1" 按钮",\n "bbox": [100, 765, 200, 805]\n }\n}\n' }
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

Reproduce link

await this.agent.aiAct(`输入 "${amount}"  并点击 确认/OK/完成`);

Reproduce Steps

await this.agent.aiAct(`输入 "${amount}"  并点击 确认/OK/完成`);