CaltechExperimentalGravity / CaltechExperimentalGravity/system_ident

[deferred] Hardware-state faults needing EPICS: suspension watchdog / Guardian / SDF

Open
#37 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

**Spin-out of #32 · Component 2 · spec §4.3.3 (item 4), §4.3.4, §4.3.8.** Folds into #28.

Covers #32 item **4**, and the class of faults it belongs to.

**Not observed.** Every run to date has been a closely supervised single-operator session, where the
operator would simply have seen a tripped watchdog. Filed so the *inferential* stopgap in Component 1
is not later mistaken for a real check.

## The fault class

Some faults are not transport faults at all. The data arrives perfectly, the excitation is commanded
correctly and the readback is healthy — but **the plant never received the drive, or changed
underneath the measurement**. Spec §4.3.4 names this class `HardwareStateFault`, distinguished from
*transport* ("we cannot get data") and *integrity* ("the data we got is wrong").

Members:

- [ ] **A suspension watchdog trips**, preventing the excitation from reaching the hardware — #32
item 4. Note the excitation readback sits **upstream** of the trip, so the commanded drive still
round-trips correctly; this is exactly why it is not detectable by #36's drive round-trip check.
- [ ] **An operator changes a filter-module switch, gain or offset mid-record.** The plant changed
under the measurement; nothing in the data path reports it.
- [ ] Guardian / lock-state transitions during a campaign.

## What Component 1 does, and what it must not claim

Component 1 can only **infer**: drive present at the excitation readback while the response RMS
collapses against the quiet-time `Pyy` baseline. It **flags** this and rejects the record. It must
not be described as detection.

**Hard constraint carried from #36:** this inference keys on coherence, and coherence must **never**
be wired to auto-abort. `safety.py:5-7` deliberately restricts the watchdog's automatic abort to
**physical hardware safety** — actuator saturation and output RMS — with coherence and fit-health as
*status* only. Inverting that would be a silent change to a deliberate safety design.

## What is actually needed

Authoritative state, which means EPICS:

- suspension watchdog state channels;
- Guardian / lock state;
- SDF monitoring, to see a filter-module change rather than infer one.

`ezca` / `pyepics` are absent from `pyproject.toml`, and writing anything through them needs operator
sign-off on exactly what may be written. That is the same boundary #28 already draws for full
filter-module snapshot/restore, so **this issue folds into #28** rather than standing alone — it is
recorded separately only so #32 item 4 has an explicit owner and is not silently treated as covered
by the Component 1 inference.

Also deferred alongside it (spec §4.3.8): **test-point ownership and arbitration between users**, the
root cause of #32 items 1 and 2. Component 1 detects that a test point was cleared; knowing *who*
holds it, and negotiating, is Component 2 (#26, #27).

---
**Campaign:** CDS hardware backend · branch [`feat/cds-hardware-backend`](https://github.com/CaltechExperimentalGravity/system_ident/tree/feat/cds-hardware-backend)
· [spec](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/docs/superpowers/specs/2026-08-03-cds-hardware-backend-design.md) · [plan](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/docs/superpowers/plans/2026-08-03-cds-hardware-backend.md) · [handoff](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/notes/cds-hardware-bringup-2026-08.md)
*Code is deferred until the plan and issues have been reviewed.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing issue #28 and the linked design spec and plan; this work is explicitly deferred and folded into #28. Read safety.py and pyproject.toml to confirm the current safety boundary and absent EPICS dependencies. Done would require the authoritative EPICS state work described here, subject to operator sign-off and the broader review.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, embedded-iot
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.