ophyd-async / FastCS API Convergence

## Summary

FastCS and ophyd-async have a lot of conceptual similarities: type hint friendly ways of making a hardware abstraction library. There is also a lot of crossover of developers from one to another, so we should make sure that similar concepts are done in the same way, and there are no "false friends" that could mislead people into making a connection that is not actually there. This report details the current state of play, and makes suggestions for how FastCS could change to be closer to ophyd-async as they both approach 1.0 releases. It was AI generated from local checkouts of both modules, then initially iterated on by myself. It is posted here for wider comment.

## 1. Background — the two APIs today

| Concept | ophyd-async | FastCS |
|---|---|---|
| Tree node | `Device` (+ pluggable `DeviceConnector`) | `BaseController` / `Controller` |
| Leaf value | `SignalR/W/RW[T]` over `SignalBackend` | `AttrR/W/RW[T, RefT]` |
| Action | `Command[P, T]`, `TriggerableCommand` | `@command()` — void/void only |
| Int-keyed group | `DeviceVector[T]` | `ControllerVector[T]` (already near-identical) |
| Declarative | class-body hints **create** children (`DeviceFiller`); metadata in `Annotated` extras (`PvSuffix`, `TangoPolling`, `Format`); *filled* now or *unfilled* until connect-time introspection (`PviDeviceConnector`, `TangoDeviceConnector`) | class-body `AttrRW(Int(), io_ref=...)` **instances** deep-copied per instance (`_bind_attrs`); hints only *validate* (`HintedAttribute`) |
| Procedural | `soft_signal_rw(...)`, `soft_command(cb)` in `__init__` | `add_attribute`/setattr in `initialise()`; IO via `AttributeIO` classes dispatched by `AttributeIORef` type |
| Value types | plain types in generics: `SignalRW[float]`, `Array1D[np.int32]`, `Table` | `DataType` instances: `Float(prec=3)`, `Waveform(np.int32, (4,))`, `Enum(cls)` |
| Serving | n/a (client) | `FastCS(controllers, transports)` → CA/PVA(+PVI), Tango, REST, GraphQL |

Existing bridge: `ophyd_async.fastcs.core.fastcs_connector(uri)` =
`PviDeviceConnector` (PVA+PVI over the network; PVI `"x"` entries already map
to `TriggerableCommand`).

---

## 2. Decisions

Agreed with Giles (maintainer of both):

1. **Shared declarative idiom = the annotation mechanism** (hints create
children; a filler provisions them at init or connect). Rule stated
identically in both projects: **class body = declarations, instance scope
= procedural construction**. FastCS class-scope Attribute prototypes and
the deepcopy half of `_bind_attrs` are removed pre-1.0. Rationale (§3.1):
the instance style must deepcopy (aliasing hazard), cannot express
"declared now, filled by introspection at connect" (why `HintedAttribute`
exists as a second mechanism), and ophyd-async *cannot* move in the other
direction because a Signal's backend depends on which connector the
instance is created with.
2. **Harsh split in FastCS**: declarative = bare hints only, made concrete at
connect/initialise time by introspection via a new `ControllerFiller`.
All IO/function attachment is procedural, in `__init__`. **The
`AttributeIORef` / IO-registration machinery is removed** (§3.2).
3. **The generic extras mechanism stays**: `ControllerFiller` yields
`(child, extras)` like `DeviceFiller` does, so protocol libraries (e.g. a
future SCPI package) can define their own `Annotated` extras. Core FastCS
defines **no** extras vocabulary for 1.0.
4. **AttributeIO survives as a per-attribute, connection-bound object**
passed as a single `io=` argument, in **R / W / RW flavours with abstract
`update` / `send` methods** so access-mode compatibility is checked
statically (§3.3). Verbs stay `update`/`send` (not getter/setter).
5. **FastCS gains typed `Command[P, T]` semantics for 1.0** (args + return).
EPICS transports stay void/void; Tango/REST/GraphQL and the embedded
connector serve typed commands fully.
6. **Embedding = ophyd-async-side `DeviceConnector`** consuming a **stable
public Controller interface** (§5), packaged behind an
**`ophyd-async[fastcs-embed]` extra**. No shared package; convergence by
convention.
7. **`DataType` classes stay** as the procedural/runtime/introspection value
(`AttrRW(Float(prec=3, units="K"))`). Classmethod factories
(`AttrRW.float(...)`) rejected: classmethod return types can't be
inherited generically (no `Self[float]`), forcing ~24 duplicated
signatures, and they produce no value usable in `Annotated` slots or by
introspection code. The `Float`/`float` declarative duplication dissolves
under decision 2 (bare hints; metadata comes from introspection); a
`Prec`/`Units`/`Shape` extras vocabulary is a post-1.0 option via the
extras mechanism.

Also agreed (answers in `prompt.md` round 3):

8. **Embedded lifecycle**: connector owns a runner (started on first
`connect_real`, idempotent); shutdown via an **atexit hook** (Giles:
"atexit is fine") plus explicit `await connector.shutdown()`;
`Device.disconnect()` upstream as a follow-up, don't block on it.
9. **Embedded + transports simultaneously** (e.g. CA GUI next to bluesky):
yes, but out of scope for the first cut; design the runner so a
transport list can be added later.
10. **FastCS gaps — both for 1.0**: (a) `AttrW` caches its setpoint (needed
for ophyd `locate()`; useful to all transports); (b) **FastCS-native
timestamps** (and severity where meaningful) on attribute updates —
Giles: useful to all transports (Tango event pushes, EPICS records),
not just the connector. Connector stamps receive-time only as an
interim until (b) lands.
11. **Typed commands over EPICS transports**: skip-with-warning (serve the
void ones, warn about the typed ones). Hard error only where the user
*declares* a typed command on something EPICS-only (matches ophyd-async
connector behaviour).
12. **Naming alignment** (while breaking pre-1.0): `prec` → `precision`;
align `min/max/min_alarm/max_alarm` with event-model `Limits` naming;
adopt `Array1D[np.int32]` / `Table` as FastCS *hint* spellings (mapping
to `Waveform`/table `DataType`s internally).
13. **Stable interface home**: formalise in fastcs core — `ControllerAPI`,
the attribute/command runtime methods, and a `ControllerRunner` (or
`Controller.serve()/stop()`) extracted from `FastCS.serve`. The ophyd
connector imports only that documented surface.
14. **Attr-from-method decorator sugar** (from the PyTango comparison,
§7.5): property-style decorators (`@attr_r` / `@attr_rw` + `.send`)
that create an Attribute bound to controller methods, so the trivial
case is one decorated getter. Pure sugar over `AttrR/W/RW` + callback
IO, built on the existing `UnboundCommand`/`UnboundScan` bind machinery
(fresh objects per instance — no prototype/deepcopy hazard). Refines
the class-body rule to: *class body = declarations + decorated
behaviour; instance scope = construction with data* (already true today
via `@command`/`@scan`).

---

## 3. FastCS redesign

### 3.1 Why annotations won (summary of the analysis)

- **Prototype aliasing**: class-scope instances must deepcopy or all
controller instances share one mutable Attribute; deepcopy is fragile
(connections/locks/bound callbacks) and costs construction time.
Annotations are inert — nothing to copy.
- **Declared-but-filled-later**: instances must be complete at class
definition; annotations natively express "exists, typed, provisioned by
introspection at connect" (`filled=False` + `check_filled`), unifying
FastCS's two mechanisms (instances + `HintedAttribute`) into one.
- **Typing**: hint == runtime by construction; extras invisible to checkers.
- **The asymmetry**: ophyd-async can't adopt instance style (backend depends
on connector choice + connect-time data), so one shared idiom is only
possible in this direction.
- **Ecosystem**: same idiom as pydantic v2 `Annotated[int, Field(...)]`.

### 3.2 Remove `AttributeIORef` / IO registration

Verified: **nothing downstream consumes `io_ref` or the IO registry** — no
transport touches them; `_connect_attribute_ios` bottoms out in
`attr.set_on_put_callback(io.send)` / `attr.set_update_callback(io.update)`,
i.e. the split is a dispatch layer over callbacks that already exist. Its
sole structural justification was that class-scope Attributes are created
before `__init__`, so they couldn't close over a connection. Decision 2
removes that situation entirely.

Delete: `AttributeIORef`, ref-type dispatch (`__init_subclass__` generic
sniffing), `ios=` kwarg, `_validate_io`, `_connect_attribute_ios`,
`_attribute_ref_io_map`, and the **second TypeVar** — `Attribute[DType_T,
AttributeIORefT]` collapses to `Attribute[DType_T]`, making `AttrRW[float]`
structurally isomorphic to `SignalRW[float]`.

### 3.3 New `AttributeIO`: per-attribute, R/W/RW flavours, single `io=` arg

```python
class ReadIO(Generic[DType_T], ABC):
def __init__(self, update_period: float | None = None): ...
@abstractmethod
async def update(self, attr: AttrR[DType_T]) -> None: ...

class WriteIO(Generic[DType_T], ABC):
@abstractmethod
async def send(self, attr: AttrW[DType_T], value: DType_T) -> None: ...

class ReadWriteIO(ReadIO[DType_T], WriteIO[DType_T], ABC): ...
```

(Exact class names to be settled in prototype — could equally be
`AttrRIO/AttrWIO/AttrRWIO` to mirror the Attr family.)

- `AttrR(dt, io: ReadIO[DType_T] | None)`, `AttrW(dt, io: WriteIO[...] |
None)`, `AttrRW(dt, io: ReadWriteIO[...] | None)` — passing a read-only IO
to an `AttrRW` is a **static** error; abstract methods force subclasses to
implement the right surface.
- `update_period` lives on `ReadIO` (it describes the IO, not the
attribute); `ONCE` still supported. `Controller.create_api_and_tasks`
schedules from `attr.io.update_period` instead of matching ref types.
- `io=None` keeps today's soft behaviour (`AttrRW` self-wires
setpoint→readback via `_internal_update`; sync-setpoint machinery
unchanged) — the analogue of `soft_signal_rw`.
- Concrete callback adapters ship for one-offs (the `soft_command`-style
escape hatch): e.g. `CallbackReadIO(update=cb, update_period=0.2)`.
- Migration is mechanical: an old IO subclass absorbs its ref's fields into
its constructor and is passed per-attribute.

### 3.4 Target: the temperature controller

Before: 3 cooperating classes (`TemperatureControllerAttributeIORef`,
`TemperatureControllerAttributeIO`, `ios=[...]` registration) + dispatch.
After — one small IO class, everything bound at creation:

```python
class TempIO(ReadWriteIO[float]):
def __init__(self, conn: IPConnection, param: str, suffix: str = "",
update_period: float | None = 0.2):
super().__init__(update_period=update_period)
self._conn, self._cmd = conn, f"{param}{suffix}"

async def update(self, attr: AttrR[float]) -> None:
resp = await self._conn.send_query(f"{self._cmd}?\r\n")
await attr.update(attr.dtype(resp.strip("\r\n")))

async def send(self, attr: AttrW[float], value: float) -> None:
await self._conn.send_command(f"{self._cmd}={value}\r\n")

class TemperatureRampController(Controller):
def __init__(self, index: int, conn: IPConnection) -> None:
super().__init__()
sfx = f"{index:02d}"
self.start = AttrRW(Int(), io=TempIO(conn, "S", sfx))
self.end = AttrRW(Int(), io=TempIO(conn, "E", sfx))
self.enabled = AttrRW(Enum(OnOff), io=TempIO(conn, "N", sfx))
self.target = AttrR(Float(precision=3), io=TempIO(conn, "T", sfx))
self.actual = AttrR(Float(precision=3), io=TempIO(conn, "A", sfx))
self.voltage = AttrR(Float(precision=3)) # soft: fed by parent @scan
```

An *introspectable* controller instead declares promised children as bare
hints and provisions them in `initialise()`:

```python
class OdinDetector(Controller):
# declared: must exist after initialise(), typed for users & pyright
frames: AttrRW[int]
acquire: Command[[], None] # typed commands, decision 5

async def initialise(self) -> None:
for name, meta in await self._query_parameter_tree():
self.filler.fill_attribute(name, ...) # exact API per prototype
# filler.check_filled() reports any promised-but-missing children
```

### 3.5 `ControllerFiller`

Mirror of `DeviceFiller` (`ophyd_async/core/_device_filler.py` is the
reference implementation — steal its shape):

- scans class hints for `AttrR/W/RW[T]`, `Command[...]`, sub-`Controller` /
`ControllerVector[T]` types; creates children unfilled; tracks
filled/unfilled; `check_filled(source)` raises listing what introspection
failed to provide.
- yields `(child, extras)` so third-party extras vocabularies work
(decision 3); core consumes none for 1.0.
- subsumes and deletes `HintedAttribute` + `_validate_type_hints`; removes
the deepcopy half of `_bind_attrs` (method binding for
`@command`/`@scan` stays).
- trailing-underscore convention adopted from ophyd-async (`stop_` →
logical name `stop`) for name clashes.

### 3.6 Typed commands

- Lift the zero-arg restriction in `Method`/`Command._validate`; keep the
captured `inspect.Signature` (already there) as the public
signature — mirrors `CommandBackend.signature`.
- ControllerAPI exposes the signature; transports declare capability:
EPICS CA/PVA serve void/void commands and **skip typed ones with a
warning** (decision 11); Tango/REST/GraphQL serve fully.

---

## 4. What changes in ophyd-async

Deliberately little (it is the reference for the idiom):

- New embedded connector (§5) behind `fastcs-embed` extra.
- `fastcs_connector(uri)` grows scheme dispatch: `pva://` →
`PviDeviceConnector` (today's behaviour), `tango://` →
`TangoDeviceConnector`, plus the in-process path taking a `Controller`
instance directly.
- Follow-up (not blocking): `Device.disconnect()` upstream discussion.

---

## 5. Embedded FastCS: `FastCSDeviceConnector`

```python
from ophyd_async.fastcs import embedded_fastcs_connector

class TempStage(Device):
ramp_rate: SignalRW[float]
power: SignalR[float]
cancel_all: TriggerableCommand
ramps: DeviceVector[TempRamp]

stage = TempStage(connector=embedded_fastcs_connector(TemperatureController(settings)))
await stage.connect() # runs controller lifecycle in-process
```

Mechanics:

- `create_children_from_annotations`: `DeviceFiller` with
`FastCSSignalBackend` / `FastCSCommandBackend` factories, `filled=False`
(same pattern as `PviDeviceConnector` / `TangoDeviceConnector`).
- `connect_real` (top level): start the `ControllerRunner` —
`initialise()`, `post_initialise()`, `create_api_and_tasks()`, controller
`connect()`, initial coros, scan tasks on the running (bluesky) loop —
then walk the `ControllerAPI` tree and fill children; `check_filled`;
`set_name`. Idempotent across reconnects. `await connector.shutdown()`
cancels scan tasks and calls `disconnect()` (decision 8).
- `connect_mock` never touches the controller → mock mode free.

Backend mappings:

| ophyd-async | FastCS |
|---|---|
| `SignalBackend.get_value` | `AttrR.get()` |
| `SignalBackend.set_callback` | `AttrR.add_on_update_callback(cb, always=True)`; stamp `time.time()`, severity 0 (decision 10b) |
| `SignalBackend.put` | `AttrW.put(value)` |
| `SignalBackend.get_setpoint` | new `AttrW` cached setpoint (decision 10a) |
| `SignalBackend.get_datakey` | from `Attribute.datatype` → `SignalMetadata` (units, precision, limits, enum choices) + `make_datakey` |
| `CommandBackend.execute` / `signature` | fastcs `Command.__call__` / `Method` signature |
| `SignalBackend.source` | e.g. `fastcs://.` |
| child Device / `DeviceVector` | sub-`Controller` / `ControllerVector` |
| (not exposed) | `@scan` methods — server-side only |

Datatype mapping: `Int/Float/Bool/String` → `int/float/bool/str`;
`Waveform(array_dtype, shape)` → `Array1D[dtype]`; `Enum(cls)` → the enum
class. **Prototype risk**: ophyd-async constrains enums to `EnumTypes`
(`StrictEnum`/`SubsetEnum`/`SupersetEnum`) while fastcs accepts any
`enum.Enum` — needs a conversion/validation decision. Same for fastcs
`Table` vs ophyd-async `Table` (pydantic-based) — map best-effort, flag
mismatches early.

The stable public Controller interface the connector may use (decision 13):
`ControllerAPI` tree (attributes, command_methods + signatures, sub_apis,
description); `AttrR.get/add_on_update_callback`, `AttrW.put` (+ cached
setpoint), `Attribute.datatype/access_mode/description/group`;
`ControllerRunner` start/stop. Nothing else — no reaching into
`BaseController` internals.

---

## 6. Migration / removal list (FastCS, pre-1.0)

- Remove: `AttributeIORef`, IO registry (`ios=`, `_validate_io`,
`_connect_attribute_ios`), `AttributeIORefT` TypeVar, `HintedAttribute`,
class-scope Attribute prototypes + `_bind_attrs` deepcopy.
- Add: R/W/RW `AttributeIO` flavours + `io=` arg; `ControllerFiller` +
extras mechanism; typed commands; `AttrW` setpoint cache;
`ControllerRunner`; renamed metadata fields (`precision`, `Limits`-aligned
names); `Array1D`/`Table` hint spellings.
- Docs: rewrite the "writing a controller" tutorial around the temperature
controller (§3.4) and an introspectable example; state the class-body /
instance-scope rule up front, identically to ophyd-async's
declarative-vs-procedural page.

---

## 7. Worked example pair (the convergence pitch)

Same developer, both projects, same shapes:

```python
# FastCS server # ophyd-async client
class Ramp(Controller): class RampDevice(Device):
target: AttrRW[float] target: SignalRW[float]
actual: AttrR[float] actual: SignalR[float]
enable: Command[[], None] enable: TriggerableCommand
# filled by initialise() introspection # filled by connector introspection
# procedural: AttrRW(Float(), io=...) # procedural: soft_signal_rw(float)
# actions: @command / typed args # actions: Command[P, T] / soft_command
```

### 7.5 The Tango pitch (FastCS vs PyTango)

FastCS must also sell as a way to write Tango Device Servers: more
compelling than PyTango, no more complicated for the simple case.

Where FastCS (with this proposal) beats PyTango:

- **One class, many faces**: Tango + EPICS CA/PVA(+PVI) + REST/GraphQL +
generated GUI from one controller (see `demo/fastcs.yaml`), plus direct
embedding in ophyd-async — PyTango gives Tango only.
- **Async-native** throughout; PyTango green modes are second-class (the
fastcs Tango DSR has to bridge our loop thread-safely already).
- **Composition**: sub-controllers/vectors vs Tango's flat namespace.
- **Introspection-driven attributes** (`ControllerFiller`) — no PyTango
equivalent.
- **Typing**: dtypes in real annotations, pyright-checked; validation with
limits/alarms in `DataType`; plain-asyncio testing (no
`DeviceTestContext`); yaml config with generated JSON schema vs Tango DB
properties.
- Decisions 5 (typed commands) and 10b (native timestamps) are
**prerequisites** for this pitch, not nice-to-haves: PyTango has typed
in/out commands and timestamped attributes already.

Where the un-amended proposal *lost* to PyTango: the trivial case. PyTango
hello-world is one decorated getter (`@attribute def current(self) ->
float: return 2.5`); the harsh split required a method + `CallbackReadIO`
adapter + `__init__` wiring. Decision 14 closes this with property-style
sugar on the existing method-binding machinery:

```python
class PowerSupply(Controller):
@attr_rw(units="V", update_period=0.5) # dtype inferred from -> float
async def voltage(self) -> float:
return await self._conn.query("V?")

@voltage.send
async def voltage(self, value: float) -> None:
await self._conn.send(f"V={value}")
```

One decorated method for the simple case — dtype stated once, in a real
annotation (better than PyTango's `dtype=` kwargs) — degrading gracefully
into `io=` objects for protocol families and the filler for introspection.
Transport-level Tango gaps noted for later (not proposal blockers):
DevState/Status conventions are currently hardcoded `ON`; event pushing
from `add_on_update_callback` should use native timestamps once 10b lands.
Docs should include a "FastCS for PyTango users" side-by-side page.

---

## 8. Work plan (PR-sized, in order)

**FastCS**
1. `AttributeIO` rework (§3.2–3.4): new R/W/RW IO classes, `io=` arg, delete
ref machinery, drop second TypeVar, move `update_period`, port demo
temperature controller + tests. *(Biggest single win; independent of the
filler.)*
2. Typed commands (§3.6): lift restriction, ControllerAPI signature,
EPICS skip-with-warning, Tango/REST/GraphQL support.
3. `ControllerFiller` (§3.5): hints-create + filled/unfilled + extras yield;
delete `HintedAttribute` + class-scope prototypes; port introspecting
controllers (odin/eiger-style) in dependent repos as validation.
4. `AttrW` setpoint cache; **native timestamps (+severity) on attribute
updates** (decision 10, 1.0 scope); `ControllerRunner` extraction from
`FastCS.serve`; document the stable interface (decision 13).
5. Naming pass (decision 12): `precision`, Limits alignment, `Array1D`/
`Table` hints.
5b. Attr-from-method decorator sugar (decision 14, §7.5): `@attr_r` /
`@attr_rw` + `.send` on the `Unbound*` bind machinery; sits on PR 1,
small and independent; include the "FastCS for PyTango users" docs page.

**ophyd-async**
6. `ophyd_async.fastcs` embedded connector (§5) + `fastcs-embed` extra +
tests against the demo temperature controller (sim serial device exists
in `fastcs.demo.simulation`). Depends on FastCS 1, 4 (setpoint cache,
runner); degrade gracefully where possible to prototype earlier.
7. `fastcs_connector` URI scheme dispatch; docs page "embedding FastCS".
8. (Follow-up) `Device.disconnect()` proposal; serve-transports-alongside
option on the runner (decision 9).

Coordination: 1–4 land in FastCS before 6 is mergeable; prototype 6 against
a FastCS branch to validate the stable interface *before* freezing it.

---

## 9. Key files for the prototyper

**ophyd-async** (`/workspaces/ophyd-async/src/ophyd_async/`)
- `core/_device.py` — `Device`, `DeviceConnector`, `DeviceVector`
- `core/_device_filler.py` — `DeviceFiller` (reference for `ControllerFiller`)
- `core/_signal_backend.py` — `SignalBackend`, `SignalMetadata`, `make_datakey`
- `core/_command.py` — `Command[P, T]`, `CommandBackend`, `TriggerableCommand`, `soft_command`
- `epics/core/_pvi_connector.py` — the unfilled-until-connect workflow
- `tango/core/_base_device.py` — introspection filling + `Annotated` extras consumption
- `fastcs/core.py` — current `fastcs_connector`
- `docs/explanations/declarative-vs-procedural.md`

**FastCS** (`/workspaces/FastCS/src/fastcs/`)
- `attributes/` — `Attribute`, `AttrR/W/RW`, `attribute_io.py`, `attribute_io_ref.py` (to be reworked/removed)
- `controllers/base_controller.py` — `_bind_attrs`, `_find_type_hints`, `_connect_attribute_ios` (bulk of the surgery)
- `controllers/controller.py` — `create_api_and_tasks` (update-period scheduling)
- `methods/` — `Method`/`Command`/`Scan` (typed-command change)
- `datatypes/` — `DataType` family (naming pass)
- `control_system.py` — `FastCS.serve` (extract `ControllerRunner`)
- `demo/controllers.py` — temperature controller (the ergonomics benchmark)

Start with the decisions in this issue and the referenced prompt.md round 3, then inspect FastCS, Controller, Attribute, AttributeIO, and FastCS.serve in the fastcs core. Map the agreed API convergence and embedding surface before changing anything; done means the pre-1.0 redesign, typed commands, stable controller interface, and related lifecycle gaps are implemented and tested.

Written by the indexing model from the issue text.