oceanbase / oceanbase/powercontext
feat(work): add advisory Jev checks for Handoff and Task Outcome evidence
@Oxidaner is already working on this.
Since Sep 19, 2026.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 212
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 199
Description
Tracking issue: #1649
Feature description
Add an opt-in, advisory Jev evaluator that checks whether the exact evidence attached to Handoff statements and Task Outcome claims supports those statements.
Priority: P1 — evidence quality for work continuity.
Problem and proposed solution
An outgoing agent says "database compatibility is verified" while the attached test output covers SQLite only. The citation can resolve correctly and still fail to support the broader claim. Passing a structural evidence check does not establish semantic support.
Resolve the exact authorized Source/Artifact evidence in PowerContext, then submit a bounded claim–evidence pair to Jev. Classify the relationship as supports, contradicts, does not address, or insufficient to judge. Keep evidence resolution, revision matching, command exit status, and other exact checks in code. Use the model only for the remaining semantic relationship.
Expose the result to the receiving agent or reviewer as an assessment with exact references and uncertainty. An unresolved or truncated source produces unknown rather than a guess. An advisory flag can request further verification; it must not accept a Handoff, approve an artifact, change a check to verified, or declare a task complete.
Acceptance criteria
- Define a bounded assessment shape with the claim identity, exact evidence references, verdict, probabilities, actual model version, policy version, and a fallback reason where applicable. Use an evaluation-owned record initially; any public contract change follows an RFC.
- Validate through public workflows using accurate claims, overbroad completion claims, partial test coverage, contradicted claims, inaccessible/missing evidence, and references to a different revision or execution attempt.
- Preserve
declaredversusverified, unknown/not-checked states, receiver acknowledgement, and all existing evidence requirements. A model verdict alone cannot upgrade them. - A timeout or provider failure leaves the underlying Handoff/Outcome unchanged and marks the assessment unavailable.
- Compare deterministic validation alone, validation plus Jev, and a suitable LLM judge using manually adjudicated cases; report unsupported-claim detection, false accusations, calibration, latency, and cost.
- Demonstrate that the assessment helps a receiving agent find a real verification gap, while task completion still relies on actual execution evidence.
Alternatives considered
Deterministic validation should handle structured test records whenever possible. Human review and a generative LLM judge remain alternatives for ambiguous or complex claims. Jev's constrained output is not proof of factual correctness.
Additional context
This is a focused evidence-quality extension to #1352 / #1419, not a new Handoff lifecycle. Reuse the evaluation and review boundaries in #1422. Do not equate an accepted receipt with completed work.
- Existing TaskCheck and TaskOutcome models
- TypeSafe citation-checking cookbook
- TypeSafe confidence guidance
Prepared with AI assistance. This is proposed work; semantic verification quality has not been measured in PowerContext.
Related Jev work: reuse transport/evaluation conventions from #1643 where useful. The coding example in #1646 is an end-to-end consumer of this assessment.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.