dgenio / dgenio/contextweaver

Enforce the dependency-closure invariant across all context pipeline stages

Open
#449 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/context complexity:average priority:medium reliability testing
Dominant language
Python
Stars
9
Forks
17
Avg merge
21h 36m
Merged PRs (30d)
22

Description

Summary

The dependency-closure guarantee ("if a selected item has a parent_id, the parent must be included in the final context") is currently applied only at stage 2 of the 8-stage context pipeline, where it adds parents to the candidate list. Later stages can still remove a parent while keeping its child. This issue proposes enforcing the invariant through to the final selected set.

Why this matters

docs/agent-context/invariants.md states the rationale directly: tool results without their tool calls produce incoherent context. The invariant is currently a candidate-generation behavior, not an output guarantee. Strengthening it makes the documented contract real and protects downstream consumers (prompt rendering, handoff packs, explanations) from orphaned children.

Current evidence

  • src/contextweaver/context/build.pyresolve_dependency_closure runs at stage 2 only; it adds parents to candidates.
  • Stage 3 (context/sensitivity.py:143-157) drops items with no parent/child awareness: a confidential tool_call parent can be dropped while its public tool_result child survives.
  • Stage 6 (context/dedup.py:44-55) removes the lower-scored of two similar items regardless of dependents.
  • Stage 7 (context/selection.py:49-68) contains no parent_id logic at all — greedy selection by score with per-kind limits and budget can drop a parent while packing its higher-scoring child.

Proposed implementation

  1. Add a post-selection closure repair step inside select_and_pack (or immediately after it in run_build_pipeline): walk parent_id chains of all selected items; for each missing parent, force-include it, charging the budget.
  2. Define an explicit eviction policy when forced parents exceed the budget: evict the lowest-scored selected items that have no dependents until the closure fits, and record the evictions in BuildStats.dropped_reasons under a new reason (e.g. "closure_eviction").
  3. Decide and document the interaction with sensitivity: a parent dropped by the sensitivity filter must NOT be re-introduced. Instead, either (a) also drop its children, or (b) keep children but mark them with a metadata flag (e.g. orphaned_by_sensitivity) so renderers can annotate. Option (a) is more consistent with the security posture; the choice should be recorded in docs/agent-context/invariants.md.
  4. Make dedup closure-aware: never remove an item that is the parent_id of a surviving item (cheap check against a parent-id index built once per build).
  5. Update docs/agent-context/architecture.md to describe closure as an end-to-end guarantee with the sensitivity exception.

Acceptance criteria

  • For any build output, every selected item with a parent_id either has its parent in the final context, or the documented sensitivity exception applies and is observable in BuildStats.
  • Dedup never removes an item that a surviving item depends on.
  • Closure-driven evictions are visible in BuildStats.dropped_reasons.
  • An invariant test exercises: parent below budget cutoff, parent removed by dedup, parent dropped by sensitivity — and asserts the documented behavior in each case.
  • make ci passes.

Test plan

  • New tests in tests/test_selection.py / tests/test_dedup.py / tests/test_build.py (or the module-matching files) covering the three scenarios above.
  • A pipeline-level property check: build over randomized parent/child chains asserts no orphaned children in ContextPack.
  • Existing golden-prompt fixtures regenerate only if output legitimately changes; review diffs.

Migration notes

Behavior change, not API change: builds that previously emitted orphaned children will now include parents (slightly higher token use) or evict children. Document in CHANGELOG.md under a "context coherence" note.

Risks and tradeoffs

  • Forced parent inclusion competes with budget; the eviction policy must remain deterministic (tie-break by ID) to preserve the determinism guarantee.
  • The sensitivity interaction is a security decision; it should follow the extra-scrutiny review rule for context/sensitivity.py.

Suggested labels

reliability, testing, area/context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read docs/agent-context/invariants.md and docs/agent-context/architecture.md, then trace resolve_dependency_closure in src/contextweaver/context/build.py through context/sensitivity.py, context/dedup.py, and context/selection.py. Run the existing tests and inspect tests/test_selection.py, tests/test_dedup.py, and tests/test_build.py before deciding the sensitivity behavior. Done means deterministic closure handling, observable closure evictions, documented behavior, pipeline-level invariant coverage, and a passing make ci.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.