Feature Request: Metacognition Hook for terraphim-skills

Feature Request: Metacognition Hook for terraphim-skills

Summary

Add a judge hook to terraphim-skills that provides structured self-correction capabilities for AI agents, based on FINAL-Bench metacognition research.

Background

FINAL-Bench (2025) found that 94.8% of performance gain comes from Error Recovery (ER) - actually detecting and correcting errors - not just saying "I might be wrong" (Metacognitive Accuracy).

Proposed Feature

Hook: judge

A pre-response hook that evaluates agent outputs for:

1. Confidence calibration
2. Error-prone patterns
3. Verification needs
4. Alternative interpretations

Interface

# terraphim-skills config
hooks:
  pre-response:
    - name: metacognition-judge
      type: judge
      config:
        min_confidence: medium
        auto_revise: false
        checklists:
          - verify_tools_used
          - check_alternatives
          - assess_confidence

Input

{
  "response": "I think this solution will work...",
  "context": {
    "task_type": "code_generation",
    "tools_used": ["rg"],
    "tools_available": ["rg", "python", "gh"],
    "session_history": [...]
  }
}

Output

{
  "judgment": {
    "confidence": "low",
    "reasoning": "Claim contains uncertainty markers without verification",
    "verification_needed": [
      "Verify with external source",
      "Check similar past errors"
    ],
    "recommended_action": "revise"
  },
  "checklist_results": [
    {
      "check": "verify_tools_used",
      "passed": false,
      "issue": "Response claims certainty but no tools verified the claim"
    }
  ]
}

Pattern Detection

The judge should detect:

Pattern	Confidence	Action
"I think..."	Low	Suggest verification
"According to X..."	High	(verified)
"Always/Never"	Low	Suggest exceptions
"Maybe/Probably"	Low	Request clarification
Tool output cited	High	(verified)

Integration with terraphim-agent

# The hook would be called automatically
terraphim-agent process --hook judge --input response.json

# Or explicitly
terraphim-agent judge "I think this will work" --context task.json

Use Cases

1. Before Responding: Judge draft response, revise if low confidence
2. During Long Tasks: Checkpoint every N minutes for error detection
3. Learning Capture: Record judgments to improve over time
4. Multi-Agent: Share judgments across meta-cortex

Benefits

• Error Reduction: Catch mistakes before they reach user
• Confidence Calibration: Match stated confidence to actual accuracy
• Learning: Track error patterns for improvement
• Trust: Users get more reliable responses

Implementation Notes

• Should be lightweight (heuristic-based, no LLM calls)
• Configurable checklists per role
• State persistence for calibration tracking
• Optional integration with terraphim memory

Related

• FINAL-Bench paper: https://huggingface.co/blog/FINAL-Bench/metacognitive
• Current workaround: Local Python implementation in skills/metacognition/

Start by reviewing the current workaround in skills/metacognition/ and the terraphim-skills hook configuration described in the issue. Then trace how terraphim-agent handles process and judge commands. Done means a lightweight, configurable judge hook supports the proposed input and output structures without requiring LLM calls.