CaltechExperimentalGravity / CaltechExperimentalGravity/system_ident
[deferred] Hardware-state faults needing EPICS: suspension watchdog / Guardian / SDF
- Dominant language
- Python
- Stars
- 0
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
**Spin-out of #32 · Component 2 · spec §4.3.3 (item 4), §4.3.4, §4.3.8.** Folds into #28.
Covers #32 item **4**, and the class of faults it belongs to.
**Not observed.** Every run to date has been a closely supervised single-operator session, where the
operator would simply have seen a tripped watchdog. Filed so the *inferential* stopgap in Component 1
is not later mistaken for a real check.
## The fault class
Some faults are not transport faults at all. The data arrives perfectly, the excitation is commanded
correctly and the readback is healthy — but **the plant never received the drive, or changed
underneath the measurement**. Spec §4.3.4 names this class `HardwareStateFault`, distinguished from
*transport* ("we cannot get data") and *integrity* ("the data we got is wrong").
Members:
- [ ] **A suspension watchdog trips**, preventing the excitation from reaching the hardware — #32
item 4. Note the excitation readback sits **upstream** of the trip, so the commanded drive still
round-trips correctly; this is exactly why it is not detectable by #36's drive round-trip check.
- [ ] **An operator changes a filter-module switch, gain or offset mid-record.** The plant changed
under the measurement; nothing in the data path reports it.
- [ ] Guardian / lock-state transitions during a campaign.
## What Component 1 does, and what it must not claim
Component 1 can only **infer**: drive present at the excitation readback while the response RMS
collapses against the quiet-time `Pyy` baseline. It **flags** this and rejects the record. It must
not be described as detection.
**Hard constraint carried from #36:** this inference keys on coherence, and coherence must **never**
be wired to auto-abort. `safety.py:5-7` deliberately restricts the watchdog's automatic abort to
**physical hardware safety** — actuator saturation and output RMS — with coherence and fit-health as
*status* only. Inverting that would be a silent change to a deliberate safety design.
## What is actually needed
Authoritative state, which means EPICS:
- suspension watchdog state channels;
- Guardian / lock state;
- SDF monitoring, to see a filter-module change rather than infer one.
`ezca` / `pyepics` are absent from `pyproject.toml`, and writing anything through them needs operator
sign-off on exactly what may be written. That is the same boundary #28 already draws for full
filter-module snapshot/restore, so **this issue folds into #28** rather than standing alone — it is
recorded separately only so #32 item 4 has an explicit owner and is not silently treated as covered
by the Component 1 inference.
Also deferred alongside it (spec §4.3.8): **test-point ownership and arbitration between users**, the
root cause of #32 items 1 and 2. Component 1 detects that a test point was cleared; knowing *who*
holds it, and negotiating, is Component 2 (#26, #27).
---
**Campaign:** CDS hardware backend · branch [`feat/cds-hardware-backend`](https://github.com/CaltechExperimentalGravity/system_ident/tree/feat/cds-hardware-backend)
· [spec](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/docs/superpowers/specs/2026-08-03-cds-hardware-backend-design.md) · [plan](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/docs/superpowers/plans/2026-08-03-cds-hardware-backend.md) · [handoff](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/notes/cds-hardware-bringup-2026-08.md)
*Code is deferred until the plan and issues have been reviewed.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing issue #28 and the linked design spec and plan; this work is explicitly deferred and folded into #28. Read safety.py and pyproject.toml to confirm the current safety boundary and absent EPICS dependencies. Done would require the authoritative EPICS state work described here, subject to operator sign-off and the broader review.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, embedded-iot
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100