ModelEngine-Group / ModelEngine-Group/nexent

[Bug] VerificationController 的错误信号正则会把 failedFileNum / planner_error 这类字段名误判为错误,即使字段值本身表示成功

Open
#3,564 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.9k
Forks
731
Avg merge
19h 34m
Merged PRs (30d)
172

Description

Nexent Version

v2.3.0

Problem Description

sdk/nexent/core/agents/verification.pyVerificationController._ERROR_RE 的定义是:

_ERROR_RE = re.compile(
    r"(traceback|exception|error:|failed|timeout|unauthorized|permission denied)",
    re.IGNORECASE,
)

这个正则没有单词边界,会匹配到 JSON 字段名里的子串,而不仅仅是真正的错误信息。
一个完全成功的工具返回结果,例如:

{"status":"COMPLETED","failedFileNum":0,"planner_error":null}

会被误判为包含错误信号——因为字段名 failedFileNum / planner_error 里含有
"failed" / "error" 子串,跟它们实际的值(表示成功)无关。

Reproduction Steps
import re
_ERROR_RE = re.compile(r"(traceback|exception|error:|failed|timeout|unauthorized|permission denied)", re.IGNORECASE)
_ERROR_RE.search('{"status":"COMPLETED","failedFileNum":0,"planner_error":null}')
# -> 命中 "failed",即使 failedFileNum 是 0、planner_error 是 null
Additional Information
相关问题:空响应误判

_looks_empty_handoff 有类似的误判:一个结构化的 handoff 里明明带着真实证据
(evidence_used、citations 等字段),只因为答案文本里同时出现了"无法列出更多结论"
这类限定性表述,就被整体判定为"空响应",即使它其实带着实质性证据。

影响

任何工具返回的 JSON 里字段名含有 "error"/"failed"(这非常常见——比如计数器字段、
可为 null 的 *_error 字段),都会导致明明成功的工具调用被 verify_after_tool_call
误判为出错,削弱了校验层的可靠性。

建议修复方向

给 _ERROR_RE 加单词边界;再加一个能识别 JSON 语义的检查,只有当 "error": 后面
的值不是 null/none/false/0/空字符串时才算真错误;_looks_empty_handoff 加一个
"实质性内容"白名单。我已经准备好修复代码和测试,会提一个 PR。

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in sdk/nexent/core/agents/verification.py by reading VerificationController._ERROR_RE, _looks_empty_handoff, and verify_after_tool_call. Reproduce the false positive with the JSON example, then inspect the existing tests or add coverage for successful field names and substantive handoffs. Done means successful tool results are not classified as errors or empty responses, while genuine error signals still are.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.