CaltechExperimentalGravity / CaltechExperimentalGravity/system_ident

Stage D: lifecycle — nothing stops an excitation on Ctrl-C; idempotent stop; snapshot/restore

Open
#16 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

**Stage D (spec §3.6, §4.2).**

### Ctrl-C leaves the AWG driving

```
grep -rn 'KeyboardInterrupt|atexit|signal\.|SIGINT|finally' src/system_ident/*.py src/system_ident/backends/*.py
-> zero matches
```

[`src/system_ident/loop.py:181`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/loop.py#L181) catches **only** `SafetyAbort`, so a `KeyboardInterrupt` during a
multi-hour `read()` propagates straight out of `run()`, past `watchdog.abort()` at
[`src/system_ident/loop.py:187`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/loop.py#L187), and the excitation keeps running. The sibling project solved this on
hardware with a `finally` + `started` flag + loud warning + conditional re-raise; that logic is correct and
hard-won and should be ported verbatim.

Fix belongs in the **backend**, not the loop — the loop must stay backend-agnostic
([`src/system_ident/backends/base.py:3`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/backends/base.py#L3)):

- `try/finally` around start → settle → fetch in `read()`, calling an idempotent `_stop_all`.
- Register `atexit` **and** a `SIGINT`/`SIGTERM` handler in `__init__` calling the same `_stop_all`
(`atexit` does not fire on `SIGTERM`).
- Add `KeyboardInterrupt` to [`src/system_ident/loop.py:181`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/loop.py#L181)'s handler as belt-and-braces, so the
campaign still runs `watchdog.abort()`.

### `ramp_down` must be idempotent

[`src/system_ident/loop.py:164`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/loop.py#L164) calls it per DoF, and `Watchdog.abort`
([`src/system_ident/safety.py:137`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/safety.py#L137)) calls it again for **every** channel at
[`src/system_ident/loop.py:187`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/loop.py#L187) — so the last DoF's channel is stopped twice. The sibling project found
that calling `stop()` in the wrong state raises "cannot join thread before it is started" and **masks the
real error**. Keep `self._started: dict[str, bool]` and no-op when not started.

### `snapshot_state` / `restore_state` are mandatory here

[`src/system_ident/loop.py:140`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/loop.py#L140) calls them unconditionally and the base class raises
([`src/system_ident/backends/base.py:66`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/backends/base.py#L66)), so a campaign dies at line 140 without them.
`TwinBackend` ([`src/system_ident/backends/twin.py:223`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/backends/twin.py#L223)) and `RTSfreerunBackend`
([`src/system_ident/backends/rtsfreerun_adapter.py:246`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/backends/rtsfreerun_adapter.py#L246)) simply save/restore their in-memory drives.

**Keep the scope honest.** Capture only what can actually be restored — which channels have a live
excitation, and their ramptime — and **state in the docstring that filter-module switch/gain/offset state
is NOT captured**. On hardware `restore_state` is the "hand control back to the damping loops" step
([`src/system_ident/safety.py:9`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/src/system_ident/safety.py#L9)), and doing that properly needs `ezca`/`pyepics` (absent from
[`pyproject.toml:21`](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/pyproject.toml#L21)) plus operator sign-off on what may be written. That is deferred to Component 2.
Do not fake it.

---
**Campaign:** CDS hardware backend · branch [`feat/cds-hardware-backend`](https://github.com/CaltechExperimentalGravity/system_ident/tree/feat/cds-hardware-backend)
· [spec](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/docs/superpowers/specs/2026-08-03-cds-hardware-backend-design.md) · [plan](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/docs/superpowers/plans/2026-08-03-cds-hardware-backend.md) · [handoff](https://github.com/CaltechExperimentalGravity/system_ident/blob/feat/cds-hardware-backend/notes/cds-hardware-bringup-2026-08.md)
*Code is deferred until the plan and issues have been reviewed.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the backend interface in src/system_ident/backends/base.py and compare the in-memory snapshot/restore implementations in twin.py and rtsfreerun_adapter.py. Then inspect lifecycle handling in the backend, loop.py, and safety.py, along with pyproject.toml to confirm hardware dependencies are absent. Done means safe interruption, idempotent stopping, and restorable in-memory excitation state without pretending to restore filter-module state.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, embedded-iot
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.