Adaptive multimodal observability gate for RF, radar, LiDAR and vision fusion

Problem

Multimodal fusion is only robust if the system detects when one modality has become unobservable or misleading. Fixed sensor weights can preserve high confidence while geometry, lighting, smoke, RF coherence, calibration, or timing assumptions have failed.

Fresh research: FIRE LIVWO, arXiv:2609.05325, submitted 2026-09-04 and accepted by IROS 2026, combines LiDAR, inertial, visual, wheel and 4D mmWave radar measurements. Its most reusable idea for RuView is online observability analysis that detects visual and geometric degeneracy and changes fusion behavior. The paper reports real underground coal mine experiments and 5.677 m average localization error for its complete system.

Source: https://arxiv.org/abs/2609.05325

Important limitation: the paper is a vehicle odometry system with hardware specific assumptions. RuView should adopt the observability contract, not copy the stack or treat the reported error as a RuView target.

Current architecture and observed limitation

RuView now has increasingly strong calibration integrity, CSI admission receipts, model expiry, timing checks, and explicit source selection. The next missing invariant is a common modality level statement of whether the current measurement geometry can constrain the requested state.

Proposed architecture

Target existing RuView sensor fusion and spatial inference packages rather than a new repository.

Add a small ObservabilityEnvelope or extend the existing capability envelope with:

• modality identifier
• observable state dimensions
• geometry condition score
• timing or synchronization quality
• calibration quality
• coherence quality where phase matters
• feature support count
• innovation or residual consistency
• degradation reason codes
• validity interval
• source receipt references
• disposition: usable, downweight, abstain

Fusion rule: degraded evidence may lower its contribution or trigger abstention, but it may never raise upstream confidence. A modality can recover only after independent valid evidence passes the configured hysteresis gate.

Initial detectors should be deterministic and physically interpretable. Examples include LiDAR plane or feature rank, radar Doppler support, CSI antenna or subcarrier support, clock skew bounds, and calibration validity.

Benchmark plan

Baseline: current fixed or existing adaptive fusion path on main.

Test both nominal and injected degradation:

• camera occlusion and low light
• LiDAR sparsity or smoke style dropout where suitable recorded data exists
• radar point sparsity and Doppler loss
• CSI packet loss and missing subcarriers
• antenna geometry changes
• clock jitter and phase coherence failure
• expired or corrupted calibration

Measure:

• localization p50 and p90
• tracking continuity and catastrophic failure count
• false confidence rate
• abstention precision and recall
• p50 and p95 fusion latency
• CPU and memory overhead
• recovery time after the failure clears

Promotion gate:

• at least 50% fewer catastrophic tracking or localization failures on the degradation suite than baseline
• nominal p90 error degradation below 5%
• false high confidence events reduced by at least 50%
• p95 added latency below 2 ms on the gateway reference platform
• all degraded states carry reason codes and source receipts
• corrupted calibration or invalid coherence causes abstention rather than confidence preservation

Security review

Evaluate adversarial RF and sensor inputs designed to manipulate observability scores, replay of previously valid envelopes, forged calibration receipts, timestamp manipulation, NaN and infinity inputs, resource exhaustion through rapid state flapping, and malicious external semantic metadata.

Fail closed on nonfinite values and unverifiable receipts. Add rate and hysteresis limits so an attacker cannot create unbounded mode switching.

Compatibility and rollback

Feature flag initially. Preserve the existing fusion path as rollback. No wire change is required unless the envelope is exposed externally; if exposed, use an additive versioned schema.

MetaHarness plan

Use the latest stable ruvnet/metaharness with a repository specific benchmark objective containing localization quality, catastrophic failure count, false confidence rate, latency and resource use. Darwin may tune bounded thresholds only after a fixed adversarial degradation manifest exists. Reject any mutation that weakens provenance, calibration, timing, privacy or nominal performance gates.

Definition of done

• ADR with failure model, options, interfaces and rollback
• deterministic observability envelope implementation
• unit and property tests
• degradation injection integration suite
• fuzz tests for serialized external envelopes if any
• ARM64 and x86 benchmark
• at least three repeated benchmark runs against current main
• focused PR with measured before and after evidence

Production classification

Production candidate after the degradation benchmark and real hardware replay gate pass. Until then: Experimental.

No specific files, tests, or entry points are named; start by locating the existing sensor-fusion and spatial-inference packages and the current fixed or adaptive fusion path. Define the observability envelope and deterministic detectors there, then add the unit, property, degradation-injection, fuzz, and ARM64/x86 benchmark coverage described in the issue. Done means the ADR, measured benchmark evidence, security checks, rollback path, and focused PR are complete.