oceanbase / oceanbase/powercontext

feat(work): add advisory Jev checks for Handoff and Task Outcome evidence

Open
#1,645 1 comment 0 reactions 1 assignee View on GitHub

@Oxidaner is already working on this.

Since Sep 19, 2026.

agent-interoperability ecosystem enhancement
Dominant language
Python
Stars
1.1k
Forks
212
Avg merge
1d 11h
Merged PRs (30d)
199

Description

Tracking issue: #1649

Feature description

Add an opt-in, advisory Jev evaluator that checks whether the exact evidence attached to Handoff statements and Task Outcome claims supports those statements.

Priority: P1 — evidence quality for work continuity.

Problem and proposed solution

An outgoing agent says "database compatibility is verified" while the attached test output covers SQLite only. The citation can resolve correctly and still fail to support the broader claim. Passing a structural evidence check does not establish semantic support.

Resolve the exact authorized Source/Artifact evidence in PowerContext, then submit a bounded claim–evidence pair to Jev. Classify the relationship as supports, contradicts, does not address, or insufficient to judge. Keep evidence resolution, revision matching, command exit status, and other exact checks in code. Use the model only for the remaining semantic relationship.

Expose the result to the receiving agent or reviewer as an assessment with exact references and uncertainty. An unresolved or truncated source produces unknown rather than a guess. An advisory flag can request further verification; it must not accept a Handoff, approve an artifact, change a check to verified, or declare a task complete.

Acceptance criteria

  • Define a bounded assessment shape with the claim identity, exact evidence references, verdict, probabilities, actual model version, policy version, and a fallback reason where applicable. Use an evaluation-owned record initially; any public contract change follows an RFC.
  • Validate through public workflows using accurate claims, overbroad completion claims, partial test coverage, contradicted claims, inaccessible/missing evidence, and references to a different revision or execution attempt.
  • Preserve declared versus verified, unknown/not-checked states, receiver acknowledgement, and all existing evidence requirements. A model verdict alone cannot upgrade them.
  • A timeout or provider failure leaves the underlying Handoff/Outcome unchanged and marks the assessment unavailable.
  • Compare deterministic validation alone, validation plus Jev, and a suitable LLM judge using manually adjudicated cases; report unsupported-claim detection, false accusations, calibration, latency, and cost.
  • Demonstrate that the assessment helps a receiving agent find a real verification gap, while task completion still relies on actual execution evidence.

Alternatives considered

Deterministic validation should handle structured test records whenever possible. Human review and a generative LLM judge remain alternatives for ambiguous or complex claims. Jev's constrained output is not proof of factual correctness.

Additional context

This is a focused evidence-quality extension to #1352 / #1419, not a new Handoff lifecycle. Reuse the evaluation and review boundaries in #1422. Do not equate an accepted receipt with completed work.

Prepared with AI assistance. This is proposed work; semantic verification quality has not been measured in PowerContext.

Related Jev work: reuse transport/evaluation conventions from #1643 where useful. The coding example in #1646 is an end-to-end consumer of this assessment.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.