feat(assessor): Add shared Jev evidence assessment after access

Status: parked until Baruch receives TypeSafe/Jev access. The user reports an access waitlist; resume after access is available.

Objective

Add an evidence assessor (assessor) to coding-policy: a shared capability that evaluates a bounded claim against specific evidence and returns a structured advisory assessment. Herdr is the first consumer; standalone agents and the central fleet reviewer can use the same implementation.

The assessor asks whether evidence supports a claim. The lead retains assignment acceptance, the reviewer retains review responsibility, and the judge resolves disputes. This capability is not a new Herdr worker pane or a replacement judge.

First slice

• Evaluate individual Herdr acceptance/completion claims against reports, cited source, and test evidence.
• Exercise the same interface on fleet policy findings, supplying the governing clause and applicable exceptions.
• Return supported, contradicted, or insufficient_evidence, plus probabilities, confidence, evidence references, model/question versions, and coverage gaps.
• Keep evidence collection, source/hash/commit validation, and workflow transitions in deterministic code.
• Start in advisory mode; missing evidence, invalid responses, and service failures must remain explicit. No automatic acceptance, review suppression, or release approval.
• Offer an offline preview of the exact selected text and a separate explicit live call.

Integration plan

• Package a shared evidence-assessor skill and executable in coding-policy; use a common contract across Herdr, standalone agents, and CI.
• Define how model-supplied semantic assessments fit rules/script-delegation.md.
• Preserve the fleet review requirement to read every rule; candidate findings supplement that review.
• Preserve Herdr's report-delivery, task-ledger, independence, judge, and current-commit verification contracts.
• Document credentials and data handling, pin the model with a renewal/evaluation procedure, and wire new code into existing diagnostics and test discovery.
• Update plugin manifest, README, and CHANGELOG; run required skill review before publication.

Evaluation before rollout

Use independently labeled reports and review findings, including unsupported completion claims, missing evidence, valid policy exceptions, conflicting evidence, and adversarial text. Measure missed problems, false alarms, uncertainty, cost, latency, and useful lead/reviewer time saved. A provider double verifies program behavior, not Jev's judgment quality. Evaluate thresholds per task and model version before permitting automation.

Later candidates: scope classification in fix loops, CI failure grouping, and relevant lesson/skill selection. These are follow-on work, outside the first slice.

Local work preserved

A preliminary Python standard-library prototype and 20 deterministic tests were written. All 20 tests passed locally. No live Jev calls ran; TYPESAFE_API_KEY was unavailable. The prototype has not completed full diagnostics, policy review, skill packaging, evaluation, or integration and is not released.

The two files are saved locally only in the coding-policy clone's Git stash:

• Stash commit: 38bbdce8ee1b298bcc670266f3f29aae838a9729
• Message: On feat/evidence-assessor: parked: evidence-assessor pending Jev access
• Original base: 09bdd5e
• Paths: skills/evidence-assessor/assessor.py and skills/evidence-assessor/tests/test_assessor.py

On a fresh feature branch, restore with git stash apply 38bbdce8ee1b298bcc670266f3f29aae838a9729. Review the prototype before continuing; it is a starting point, not an accepted design or implementation. This issue preserves the design even if work resumes on another machine without the local stash.

References

• TypeSafe API
• Claim/citation checking pattern
• Confidence semantics
• Model versions
• Jev 1.13 limitations

Begin by reviewing the local prototype in skills/evidence-assessor/assessor.py and its 20 deterministic tests in skills/evidence-assessor/tests/test_assessor.py, restoring stash 38bbdce8ee1b298bcc670266f3f29aae838a9729 if needed. Read the referenced TypeSafe API material and existing policy and diagnostics guidance before settling the contract. Done means reviewed design, access-dependent evaluation, policy review, packaging, integration, and required tests and diagnostics are complete.