CaltechExperimentalGravity / CaltechExperimentalGravity/system_ident
Per-record integrity verdict + excitation-readback-vs-commanded drive round-trip
- Dominant language
- Python
- Stars
- 0
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
**Spin-out of #32 · Stage D (#14, #15, #16) · spec §4.3.6, §4.3.2.**
Covers #32 items **1, 5** and part of **9**.
**Nothing here has been observed** — every run to date has been a closely supervised single-operator
session. This is the seam that replaces "a human was watching."
## 1. The scattered asserts become one verdict
#14 already requires `read()` to assert integer-second duration, exact returned length, and an integer
multiple of `nperseg_hw`. Make that **one enumerable verdict object** rather than scattered
`assert`s, so the checks are listable, reportable to the dashboard, and testable one at a time. It
grows to cover:
- [ ] every block **adjacent** to the previous one (#35's chunked read) — no gap, ever;
- [ ] length exactly `round(duration * fs_hw)` **per channel**, with `fs_hw` now per-channel (#34);
- [ ] all samples **finite**;
- [ ] `start_time` vs requested, sample count vs duration, probed rate stable across the campaign,
GPS monotonic — #32 item 9;
- [ ] **drive round-trip**: the excitation readback vs the commanded array — below.
## 2. The drive round-trip check
Compare the `:-_..._EXC` readback against the array actually commanded. This is
newly known to be feasible: spec §8b measured that readback as NDS-readable at the model rate,
round-tripping amplitude and shape at float-level residuals on real hardware.
It is the direct detector for three separate things:
- an **injection test point cleared by another user or process** — #32 item 1;
- **large / NaN / underflow in the excitation** reaching the front end — #32 item 5 (its
*pre*-injection counterpart folds into #4's `_check_drive_limits`, not a second mechanism);
- a **stream underrun** on #31's default `ArbitraryStream` path — a gap in the excitation.
**This matters because `H_err` and coherence cannot see any of it.** #8 measured a 200%-wrong FRF
reporting median coherence **1.00000**: both are period-to-period scatter, so a common-mode corruption
is invisible to them. An independent check is the only thing that catches this class. #8's other
independent check — measured `Xbar` phase against the known injected multisine phase — is the natural
companion; use both rather than inventing a third.
## 3. Reject vs abort
- **Integrity faults** — `DataIntegrityError`, `TimingFault`, a short or non-finite block, any gap —
**reject the record.** It contributes **zero weight**.
> Never a *small* weight. This is #6's lesson restated: `loop.py:444` zeroing `var_H`, plus the
> `1e-9` floor at `:460`, turns "exclude this" into a measured weight of **4.56e+19** that swamps
> every subsequent pass for the rest of the campaign. Exclude with `inf`.
- **Transport and hardware-state faults** — `TestpointLost`, `TransportUnavailable`,
`TestpointTimeout`, `HardwareStateFault` — **abort the campaign**, through the *existing* path:
#16's idempotent `_stop_all` in its `finally`, then `watchdog.abort()`. No new abort mechanism and
no second teardown path to keep in sync. #16's `finally` therefore covers **every** transport fault,
not just `KeyboardInterrupt`.
### Guard rail — stated because it will otherwise be got wrong
Data-integrity and coherence faults must **never** enter `Watchdog.evaluate`'s breach list.
`safety.py:5-7` deliberately keys automatic abort on **physical hardware safety only** — actuator
saturation and output RMS — with coherence and fit-health surfaced as *status*. #32 item 4 (a
suspension watchdog trip) is detectable only inferentially *from* coherence, so without this rule
someone will reasonably wire it to auto-abort and silently invert that design decision. Faults surface
via the verdict and the dashboard, not as `SafetyReport.breaches`.
## 4. Re-take is asymmetric, and the cache must respect it
- A gap or fault in a record containing **one or more excitations** is a failure mode. The record is
dead, and re-taking it is a **new injection needing its own operator approval** (Rule 2, #17).
Never automatic.
- A **passive** measurement — the quiet-time background read that yields `Pyy` (#18) — carries no
excitation, so it **may be re-attempted a limited number of times**, bounded and logged.
- A **rejected record must not enter #15's read cache**, and must invalidate that generation's entry.
#15's rationale is upgraded accordingly: for a test point there is no second chance to fetch, so the
cache is a correctness affordance and not only a 7.0 h → 2.5 h speed-up.
## 5. Decimate once, after assembly
Software decimation (hardware rate → `fs`) is applied to the **assembled contiguous record**, never
per block: per-block decimation would reintroduce #10's measured edge failure (6.9 % on the first
period, 53 % on the last) at every chunk boundary. Because gaps are a hard failure there is never a
seam to decimate across.
## Verification
All on the #5 fake harness: each fault yields the right verdict entry and the right reject-vs-abort
outcome; the excitation is stopped **exactly once**; a rejected record contributes exactly zero weight
to `_accumulate`; and a commanded-vs-readback mismatch is caught in a case where coherence still
reports ~1.
---
**Campaign:** CDS hardware backend · branch [`feat/cds-hardware-backend`](https://github.com/CaltechExperimentalGravity/system_ident/tree/feat/cds-hardware-backend)
· [spec](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/docs/superpowers/specs/2026-08-03-cds-hardware-backend-design.md) · [plan](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/docs/superpowers/plans/2026-08-03-cds-hardware-backend.md) · [handoff](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/notes/cds-hardware-bringup-2026-08.md)
*Code is deferred until the plan and issues have been reviewed.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked design plan and spec, then inspect loop.py:444 and :460, safety.py:5-7, and the #5 fake harness. The verification should cover verdict entries, reject-versus-abort behavior, exactly-once excitation stopping, zero contribution from rejected records, and detection of a commanded/readback mismatch despite high coherence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100