SOTA: counterfactual-causal skill graph for compact executable skill retrieval

Finding

CaSKG, arXiv:2608.25500, submitted 2026-08-26, treats skill retrieval as a causal graph calibration problem rather than independent vector similarity. It builds a high-recall directed candidate graph from semantic, lexical, input/output, and structural evidence, then uses direction-conditioned counterfactual probes that remove, substitute, and reorder skill pairs before publishing a state-filtered weighted graph. Across six LLMs, the originating team reports the best result in all 12 model/benchmark combinations: ScienceWorld macro average 72.62 to 80.50 and ALFWorld success 80.01% to 86.79%, with fewer mean environment steps. Code is public.

Evidence class: originating-team report with code, not independently reproduced by RuV.

Weighted Opportunity Score: 4.31/5.

Why this matters to PIR

PIR already has RuVector HNSW, graph, memory, Dream Machine evaluation, and skill/harness evolution work. The missing reusable primitive is calibrated directed dependency between skills. Pure nearest-neighbor retrieval can return individually relevant skills in the wrong procedural order or omit prerequisite and verification skills.

Proposed RuVector primitive

Add an experimental CausalSkillGraph above existing vector retrieval rather than replacing HNSW:

1. HNSW provides high-recall candidate skills.
2. Existing graph primitives store directed candidate edges.
3. Edge evidence carries semantic, input/output, structural, repair, and counterfactual support.
4. Edge confidence is Bayesian-smoothed and provenance-linked.
5. Runtime expansion is filtered by current state from Core Memory or WorldGraph.
6. Retrieval returns a compact ordered skill subgraph with prerequisites and verification steps.

No LLM-generated edge may become authoritative solely because a judge says it is causal. Counterfactual evidence and successful execution traces remain evidence; promotion is benchmark-gated.

Reuse

The same primitive can serve Dream Machine failure-localized skills, Ruflo agent skills, RuView procedures, MetaHarness generated capabilities, and LatentMesh protocol skills.

Benchmark

First reproduce one CaSKG condition with the public code and pinned commit. Then compare RuVector vector-only, naive graph expansion, and causal-calibrated expansion under identical models and task splits. Report success, steps, retrieval tokens, graph-build cost, counterfactual probe cost, edge precision, stale-edge behavior, and ablations.

Falsification

Reject if a simpler prerequisite schema matches the result, if graph construction cost dominates repeated use, if edges do not transfer across models or environments, or if state filtering creates brittle failures under missing state.

Acceptance

Require at least 5 absolute points task success improvement or 20% fewer environment/tool steps over the stronger existing RuVector retrieval baseline on held-out tasks, with retrieval context no larger than baseline, no protected-slice regression above 2 points, full edge provenance, and deterministic rollback to vector-only retrieval.

Related program: #837.

Start by reproducing one CaSKG condition with its public code and pinned commit. Then compare vector-only, naive graph, and causal-calibrated retrieval under identical models and task splits, measuring the metrics and ablations listed in the issue. Done means meeting the stated success or step-reduction threshold, preserving provenance and rollback, and passing the protected-slice and context-size constraints.