Version and regression-test all model-facing text surfaces (card formats, plugin prompts, meta-tool descriptions)

Summary

Introduce explicit versioning for every text surface a model consumes — the ChoiceCard render format, the LLM-plugin prompt templates, the gateway meta-tool names/descriptions, and the call-prompt/handoff render templates — with version stamps in result metadata and golden regression tests per surface.

Why this matters

These surfaces are prompts: changing a card line format or a summarizer instruction changes model behavior in every downstream deployment. Versioning makes such changes visible, reviewable, and traceable (which prompt produced this stored summary?), enables A/B comparison and audits, and protects integrators from silent behavior drift between releases. This is the prompt-engineering analogue of the project's existing schema-drift gate (make schemas-check).

Current evidence

• src/contextweaver/extras/llm_summarizer.py lines 43–52: DEFAULT_SUMMARY_PROMPT / extractor prompts are unversioned constants; no prompt_version is recorded on ResultEnvelope (src/contextweaver/envelope.py) or FirewallStats.
• src/contextweaver/routing/cards.py — render_cards_text() line format ([{idx}/{total}] {id} ({kind}) — {description}) has no format identifier and no golden test of its own (demo casts in make record-demos-check cover it only incidentally).
• src/contextweaver/adapters/mcp_gateway.py — tool_browse/tool_execute/tool_view descriptions are the instructions client LLMs actually follow, and are similarly unversioned.
• grep -rn "prompt_version" src/ returns no matches.

Proposed implementation

1. Add a small registry module (e.g. prompt_surfaces.py) declaring (surface_name, version, text-or-format) for: card render format, summary prompt, fact-extraction prompt, meta-tool descriptions, call-prompt template (context/call_prompt.py), handoff render (context/handoff.py).
2. Stamp versions where output is persisted: FirewallStats.prompt_version for LLM-path summaries, ResultEnvelope.metadata, GraphManifest/RouteTrace for the card format version.
3. Add golden tests per surface: fixed inputs → committed expected text, failing on any unreviewed change (pattern mirrors make record-demos-check).
4. Add a changelog policy: bumping a surface version requires a CHANGELOG entry and, where the benchmark can detect it, a benchmark-delta note.
5. Documentation page listing current surface versions and their change history.

Example prompt, schema, or interface

SURFACES = {
    "choice_card_text": SurfaceVersion("v1", renderer=render_cards_text),
    "firewall_summary_prompt": SurfaceVersion("v1", text=DEFAULT_SUMMARY_PROMPT),
    "gateway_tool_browse_desc": SurfaceVersion("v1", text=TOOL_BROWSE_DESCRIPTION),
}
# FirewallStats(..., prompt_version="firewall_summary_prompt/v1")

Acceptance criteria

• Every listed surface has a declared version and a golden regression test.
• LLM-path summaries persist the prompt version that produced them.
• Changing any surface text without bumping its version fails CI.
• A docs page enumerates surfaces and versions.

Test and evaluation plan

• Golden tests per surface (committed expected outputs).
• Serde round-trip tests for the new metadata fields.
• CI drift check analogous to schemas-check (could be folded into make ci).

Migration notes

New optional metadata fields on FirewallStats/ResultEnvelope — additive, with to_dict/from_dict defaults so previously serialized data loads unchanged. Contributor workflow gains one rule: surface edits require version bumps.

Risks and tradeoffs

Slight contributor friction (golden tests must be regenerated deliberately). Version stamps add a few bytes to persisted envelopes. The registry must not become a kitchen sink — limit it to genuinely model-facing surfaces.

Suggested labels

prompt-engineering, reliability, evals, area/context

Start by reading src/contextweaver/extras/llm_summarizer.py, envelope.py, routing/cards.py, adapters/mcp_gateway.py, context/call_prompt.py, and context/handoff.py, then inspect make record-demos-check and schemas-check. Map each listed model-facing surface to its persisted metadata and golden test. Done means every surface is versioned, regression-tested, serialized compatibly, documented, and checked by CI.