[Parked pending adopter evidence] First-class pinned context items

Summary

Introduce a pinned: bool (or priority: pinned) flag on ContextItem so callers can mark items (active task statement, standing user instructions, safety policies) that must be included in every build for their phase, before budget-driven selection runs.

Why this matters

Today every candidate competes on the same score (recency, tag match, kind priority, token cost in context/scoring.py). Agent builders routinely have a small set of items that must never be dropped — the current task, a system policy, a user preference. Without pinning, users over-inflate budgets or re-ingest items to keep them alive, which defeats deterministic budgeting and makes behavior fragile across sessions.

Current evidence

• src/contextweaver/types.py — ContextItem has kind, tags, sensitivity, parent_id, but no pin/priority override field (verified by grep for pin/pinned across src/contextweaver/context/ and config.py: no matches).
• src/contextweaver/context/selection.py — greedy budget packing over scored candidates only.
• ItemKind.policy exists (types.py:42) and gets a kind-priority weight, but a high kind priority is still probabilistic, not a guarantee.
• docs/agent-context/invariants.md — dependency closure already implements a "must include" path (parents of selected children), so the pipeline has precedent for guaranteed inclusion.

External context

Agent frameworks commonly expose "system"/"core memory" blocks that are always in context (e.g. Letta/MemGPT core memory; provider system prompts). A deterministic pinning primitive is table stakes for a context compiler.

Proposed implementation

1. Add pinned: bool = False to ContextItem (types.py) with to_dict/from_dict support and schema regeneration (make schemas).
2. In context/selection.py, partition candidates into pinned and scored pools; pack pinned items first (still subject to the sensitivity filter and firewall — pinning must not bypass sensitivity_filter, stage 3).
3. Define overflow semantics: if pinned items alone exceed the phase budget, raise BudgetExceededError with a clear message listing offending IDs (fail loudly, deterministic).
4. Record pinned inclusions in BuildStats (e.g. pinned_included, and a dropped_reasons guard test proving pinned items never appear there).
5. Document interaction with dependency closure (pinned children pull parents as usual).

AI-agent execution notes

• Inspect first: src/contextweaver/context/selection.py, scoring.py, build.py, types.py, envelope.py (BuildStats), docs/agent-context/invariants.md (stage order is fixed — pinning must be a selection-stage concern, not a new stage).
• Run make ci (schemas-check will fail until make schemas regenerates schemas/ and docs/schemas/v0/).
• Tests: extend tests/test_selection.py, tests/test_manager.py; add a regression that a pinned confidential item is still dropped by the sensitivity floor.
• Do not change stage order or weaken sensitivity enforcement; do not let pinning bypass the firewall.

Acceptance criteria

• A ContextItem(pinned=True) ingested item appears in every build()/build_sync() output for its phase while budget allows.
• Pinned items never appear in BuildStats.dropped_reasons for budget/dedup reasons.
• Sensitivity filtering still applies to pinned items (test included).
• Pinned-overflow raises BudgetExceededError naming the offending item IDs.
• JSON schemas and AGENTS.md key-types table updated.

Test plan

Unit tests for selection partitioning, overflow error, sensitivity interaction, serde round-trip; property check that adding a pinned item never reduces determinism (same inputs → same pack); make ci green including schemas-check.

Documentation plan

docs/concepts.md (new "Pinned items" entry), docs/quickstart.md snippet, CHANGELOG.md, AGENTS.md module/type tables if fields change.

Migration and compatibility notes

Additive field with default False; older serialized payloads round-trip via existing missing-key defaults. Not expected to require migration.

Risks and tradeoffs

Pinning can become a crutch that erodes budget discipline — docs should frame it as "small, rare, audited." Alternative considered: a very high kind-priority weight (rejected: probabilistic, not guaranteed).

Suggested labels

product, architecture, reliability

Start with src/contextweaver/types.py, context/selection.py, scoring.py, build.py, and envelope.py, then read docs/agent-context/invariants.md for the fixed stage order. Extend tests/test_selection.py and tests/test_manager.py, including overflow, sensitivity, serde, and determinism cases; run make schemas and make ci. Done means pinned items are selected deterministically without bypassing sensitivity, overflow names item IDs, stats record inclusions, and schemas and documented type tables are updated.