Escalation/capability judges: calibration knobs for tool-step (agentic) responses

Context

The escalation judge's packaged prompt and the capability classifier's rubric read like they were calibrated on final answers from coding agents (verifiers, harnesses, test passes). When libsy runs on the live step path of a general agent runtime, most judged responses are intermediate tool steps: a couple of tool calls, little or no prose, success only visible in the tool results.

Ask

Knobs (or guidance) for calibrating the judges on tool-step responses, for example:

• a documented way to extend or re-anchor the escalation judge's rubric for intermediate steps without replacing the whole packaged prompt (today the prompt override is all-or-nothing);
• optionally letting the escalation judge see recent tool results (not just message text) the way the stage router's signal extractor does — failure text in tool results is the strongest escalate evidence we have in practice;
• published calibration notes: what verdict base rates the defaults were tuned for, so hosts can tell drift from miscalibration.

We run escalation mode per agent step in production behind a flag (fork: gumloop/gumloop-switchyard) and can share verdict distributions from our replay harness if that helps ground the defaults.

Start by locating the escalation judge's packaged prompt and override handling, then compare the capability classifier rubric with the stage router's signal extractor. Review the replay harness and determine how tool results and verdict distributions could be represented; done means the requested calibration guidance or knobs are defined without assuming an all-or-nothing prompt replacement.