web-infra-dev / web-infra-dev/midscene
[Bug]: Using version 1.10.5, the issue frequently occurs where the JSON format returned by the LLM is incorrect, causing midscene to report errors and fail.
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 14.9k
- Forks
- 1.2k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 96
Description
Version
System:
OS: Windows 11 10.0.26200
CPU: (8) x64 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
Memory: 2.10 GB / 15.71 GB
Browsers:
Chrome: 150.0.7871.116
Edge: Chromium (150.0.4078.65)
Internet Explorer: 11.0.26100.8115
npmPackages:
@midscene/cli: ^1.10.5 => 1.10.5
@midscene/web: ^1.10.5 => 1.10.5
Details
Reproduce link
无
Reproduce Steps
LLM信息
MIDSCENE_MODEL_NAME=qwen3.7-plus
MIDSCENE_MODEL_FAMILY=qwen3
发送的是点击事件
ai-call.log
[2026-07-20T19:21:57.585+08:00] adapter chat completion params: {"config":{"temperature":0,"enable_thinking":false}}
[2026-07-20T19:21:57.585+08:00] sending request to qwen3.7-plus
[2026-07-20T19:22:00.080+08:00] response reasoning content:
[2026-07-20T19:22:00.080+08:00] response content: ```json
[
"bbox": [345, 529, 391, 565],
"errors": []
}
[2026-07-20T19:22:00.081+08:00] adapter chat completion params: {"config":{"temperature":0,"enable_thinking":false}}
[2026-07-20T19:22:00.082+08:00] sending request to qwen3.7-plus
[2026-07-20T19:22:00.245+08:00] response reasoning content:
[2026-07-20T19:22:00.245+08:00] response content: ```json
[
"bbox": [345, 529, 391, 565],
"errors": []
}
ServiceError: failed to locate element:
failed to parse LLM response into JSON. Error - Error: Unexpected character "`" at position 59. Response -
[
"bbox": [345, 529, 391, 565],
"errors": []
}
at Service.locate (D:\project_code\midscene-monitor\node_modules\@midscene\core\src\service\index.ts:181:13)
at processTicksAndRejections (node:internal/process/task_queues:104:5)
at Object.executor (D:\project_code\midscene-monitor\node_modules\@midscene\core\src\agent\task-builder.ts:551:28)
at TaskRunner.flush (D:\project_code\midscene-monitor\node_modules\@midscene\core\src\task-runner.ts:335:25)
at TaskExecutor.runPlans (D:\project_code\midscene-monitor\node_modules\@midscene\core\src\agent\tasks.ts:349:20)
at PuppeteerPageAgent.callActionInActionSpace (D:\project_code\midscene-monitor\node_modules\@midscene\core\src\agent\agent.ts:697:24)
at D:\project_code\midscene-monitor\node_modules\@midscene\core\src\agent\agent.ts:719:7
at PuppeteerPageAgent.aiTap (D:\project_code\midscene-monitor\node_modules\@midscene\core\src\agent\agent.ts:718:5)
at ScriptPlayer.playFlowItem (D:\project_code\midscene-monitor\node_modules\@midscene\core\src\yaml\player.ts:585:7)
at ScriptPlayer.playTask (D:\project_code\midscene-monitor\node_modules\@midscene\core\src\yaml\player.ts:308:9)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at src/service/index.ts:181, where the stack trace reports the LLM response parsing failure, and inspect the response handling used by the locate flow. Reproduce with the logged fenced response if possible, then determine the expected object shape and how malformed JSON should be handled. Done means the reported response no longer causes an unhandled parsing failure and the behavior is covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100