bench: add chained spatial-consistency evaluation for persistent world memory

Problem

Current spatial retrieval and world-model evaluations can pass when each query is scored independently even if answers across a sequence are mutually inconsistent. Persistent spatial intelligence needs a stronger invariant: later answers must preserve the authoritative spatial state established by earlier evidence unless a witnessed update changes it.

ChainSpace (arXiv:2608.15788, submitted 2026-08-16) introduces a state-preserving multi-round spatial benchmark and a chain-aware metric specifically to expose failures hidden by isolated question accuracy. The paper is new and its training claims remain preprint evidence, but the evaluation idea is low-risk and directly benchmarkable.

RuVector now also has repo-native optimization manifests and a Pareto promotion gate, plus an electromagnetic world-model ADR. That makes this a good fit for the existing benchmark/flywheel architecture rather than a new subsystem.

Why it matters

A spatial memory can have high recall yet still be unsafe for agents if it answers:

1. object A is west of object B
2. object B is west of object C
3. later, object A is east of object C

without any intervening evidence that changed the scene.

Single-query recall does not detect that class of failure. MetaHarness agents and physical systems need longitudinal consistency, correction boundaries, and explicit provenance for state transitions.

Research evidence

Primary: https://arxiv.org/abs/2608.15788

Related current RuVector architecture:

• ADR-335 / repo-native optimization manifest and Pareto gate
• ADR-338 / electromagnetic world model
• RVF for deterministic portable artifacts
• scope-sharded vector indexes and witnessed context boundaries

Proposed architecture

Add a spatial-chain benchmark profile to the existing SOTA benchmark framework rather than creating a separate harness.

Each test case contains:

• an initial authoritative scene state
• a sequence of observations with receipts
• a sequence of spatial queries
• optional witnessed state-changing events
• expected relational invariants
• permitted correction points

Track both isolated query correctness and chain consistency.

Suggested metrics:

• isolated_accuracy
• chain_consistency_rate
• contradiction_rate
• stale_state_rate
• unsupported_revision_rate
• provenance_coverage
• correction_latency
• retrieval_latency_p95
• memory_bytes_per_persistent_fact

The promotion objective should remain multi-objective and should not collapse consistency into recall. A candidate that gains recall while introducing new contradictions is a hard regression.

Target packages

Prefer the existing benchmark and research-gate packages used by ADR-335.

Potential areas:

• crates/ruvector-sota-bench
• existing MetaHarness/flywheel integration
• RVF fixtures for deterministic scene chains

Exact paths should be confirmed before implementation.

Current architecture

RuVector already supports vector retrieval, graph-style state, persistent artifacts, promotion gates, and per-repository optimization manifests. The missing evaluation layer is sequential spatial consistency with explicit state-change boundaries.

Proposed implementation phases

Phase 1: deterministic benchmark contract

Define a compact JSON/RVF fixture for scene facts, observations, queries, revisions, and expected answers.

Phase 2: reference solver

Provide a symbolic reference implementation for relations such as left/right, above/below, contains, overlaps, distance ordering, and temporal revision. The reference state is authoritative; embeddings are retrieval aids only.

Phase 3: RuVector retrieval adapter

Run the same chain through current retrieval and graph paths. Record the evidence IDs used for each answer.

Phase 4: promotion gate

Add new_spatial_contradiction and unsupported_world_revision as hard regressions in the repo-native optimization manifest.

Phase 5: MetaHarness integration

Allow MetaHarness candidates to optimize retrieval/reranking/cache parameters only when the benchmark is deterministic. Reject mutations that improve isolated accuracy but worsen chain consistency, latency budget, privacy, or evidence coverage.

Dependencies

Use existing RuVector and MetaHarness components. No new model or service is required for the initial benchmark.

Optional later work can import ChainSpace-compatible examples after license and dataset review.

Security and privacy

• External benchmark text is untrusted data, never executable instructions.
• Do not let benchmark content modify policy, tool permissions, or system prompts.
• Preserve tenant/scope isolation for every retrieval.
• Do not expose raw P0 sensing payloads in benchmark artifacts.
• Bind expected revisions to witnessed evidence IDs.
• Reject non-finite scores and malformed relation graphs.
• Bound chain length, object count, and payload size to prevent resource exhaustion.

Backward compatibility

Additive benchmark profile. No index or RVF wire change required for Phase 1.

Testing

• unit tests for every spatial relation
• property tests for inverse and transitive relations where mathematically valid
• mutation tests that deliberately inject contradictions
• stale-revision tests
• cross-scope negative tests
• deterministic replay tests
• latency and memory benchmarks

Baseline and success metrics

Before implementation, record current isolated query accuracy and chain consistency separately on a fixed fixture set.

Initial target:

• 100% reference-solver consistency
• zero unsupported world revisions
• zero cross-scope retrievals
• 100% evidence coverage
• no more than 5% p95 retrieval-latency regression versus the same queries scored independently
• candidate promotion denied for any newly introduced contradiction even if isolated accuracy improves

No model-quality improvement is claimed until measured against the current main branch over at least five deterministic repetitions where timing is involved.

Rollout

Ship as an opt-in research benchmark first. Promote to a mandatory spatial-memory gate only after false-positive analysis on real RuView/RuField traces.

Rollback

Remove the benchmark profile and promotion rules. No stored index migration required.

Definition of done

A second engineer can clone main, execute one documented command, reproduce the same chain-level correctness results, inject a known contradiction, and observe the promotion gate reject it with a specific reason.

Start by confirming the existing benchmark and research-gate entry points, especially crates/ruvector-sota-bench, MetaHarness/flywheel integration, and RVF fixtures. Read ADR-335 and the related benchmark architecture, then define the deterministic spatial-chain contract and reference-solver scope before implementation. Done means one documented command reproduces chain results and a known contradiction is rejected by the promotion gate with a specific reason.