Teensy real-time communication

Dot Prediction → Teensy Trigger Rig (direct WebSocket)

Sending reveal events (predicted point, actual point, error, reaction time) from a
JavaScript dot-prediction task running in Chrome on an iPad to a Teensy 4.1,
which strobes a digital trigger to the DAQ for neural time-stamping.

Task page is served over plain HTTP, so ws:// is clean (no mixed-content block).
No CORS work needed — the WebSocket handshake doesn't use CORS.

---

Core design principle

Separate the timing trigger from the metadata. They have different requirements
and serving both with one mechanism is where these rigs go wrong.

• Trigger — low-bandwidth, timing-critical digital event the DAQ timestamps in neural
  time. Answers only "a reveal happened, here's its code." Fire it as fast as possible.
• Metadata (pred, actual, error, RT) — high-bandwidth, not timing-critical. Logged,
  keyed by a shared counter, merged offline.

So on each packet the Teensy: fires the trigger first, then deals with metadata.
Packet layout makes the code readable without parsing:

[uint8 code][JSON metadata...]

Teensy reads byte 0 → strobes → only then parses the JSON tail.

---

Topology

iPad Chrome (WS client, WiFi)  ─┐
                                ├─ same LAN / subnet ─→ Teensy 4.1 (WS server) ─→ DAQ trigger
your HTTP page server ──────────┘

Physical requirement for direct-to-Teensy: the iPad must be able to route to the
Teensy's IP. iPad is on WiFi, Teensy is on wired Ethernet, so a router/AP must bridge both
onto one subnet — Teensy in a LAN port of the same router the iPad's WiFi hangs off, both on
192.168.1.x.

Lower-jitter option: a USB-C-to-Ethernet adapter puts the iPad on the same wired switch.
Same code, less WiFi jitter.

---

Message format

{
  "seq": 1423,        // monotonic counter — offline merge key + dropped-packet detection
  "code": 4,          // event type -> the digital trigger value
  "t_js": 128374.51,  // performance.now(), browser clock (ms) — never touches neural time
  "trial": 87,
  "pred":   [x, y],
  "actual": [x, y],
  "error":  12.3,
  "rt":     843.2
}

• seq is the glue for offline merging and the drop detector.
• t_js is browser-clock only; it reaches neural time only through the hardware trigger.

---

Library choice (the one thing to verify)

Teensy needs a WebSocket server layer on top of QNEthernet's TCP. Pragmatic pick:
Links2004/arduinoWebSockets (WebSocketsServer) — bundles handshake, framing, masking,
ping/pong.

Compile risk to check against current docs: arduinoWebSockets selects its network backend
at compile time (WEBSOCKETS_NETWORK_TYPE); binding it to QNEthernet's
EthernetServer/EthernetClient (in the qindesign::network namespace) is the fiddly bit.
Two routes:

1. Point it at the generic-Ethernet backend and let QNEthernet's Arduino-compatible types satisfy it, or
2. Fall back to NativeEthernet, whose API matches what the library expects out of the box.

If the library fights QNEthernet, NativeEthernet is the path of least resistance for a
first working version. Verify before committing wiring — this is the only spot where "it just
compiles" isn't guaranteed.

---

Teensy firmware sketch

#include <QNEthernet.h>
#include <WebSocketsServer.h>          // Links2004/arduinoWebSockets
using namespace qindesign::network;

// trigger out — 8-bit parallel + strobe (match to your DAQ digital port)
const uint8_t CODE_PINS[8] = {2,3,4,5,6,7,8,9};
const uint8_t STROBE = 10;
const uint16_t STROBE_US = 100;

IPAddress ip(192,168,1,50), subnet(255,255,255,0), gw(192,168,1,1);
WebSocketsServer ws(8080);

inline void setCode(uint8_t c){
  for (uint8_t i = 0; i < 8; i++) digitalWriteFast(CODE_PINS[i], (c >> i) & 1);
}

inline void fireTrigger(uint8_t code){
  setCode(code);
  digitalWriteFast(STROBE, HIGH);
  delayMicroseconds(STROBE_US);
  digitalWriteFast(STROBE, LOW);
}

void onWsEvent(uint8_t num, WStype_t type, uint8_t* payload, size_t len){
  if (type == WStype_BIN && len >= 1){
    fireTrigger(payload[0]);            // FIRE FIRST — byte 0 is the code
    uint32_t t = micros();              // receipt time, validation only
    Serial.write(payload + 1, len - 1); // JSON tail -> log seq/pred/actual/error/rt
    Serial.printf("  t_us=%lu\n", t);
  } else if (type == WStype_CONNECTED){
    Serial.printf("client %u connected\n", num);
  } else if (type == WStype_DISCONNECTED){
    Serial.printf("client %u disconnected\n", num);
  }
}

void setup(){
  Serial.begin(115200);
  for (uint8_t i = 0; i < 8; i++){ pinMode(CODE_PINS[i], OUTPUT); digitalWriteFast(CODE_PINS[i], LOW); }
  pinMode(STROBE, OUTPUT); digitalWriteFast(STROBE, LOW);

  Ethernet.begin(ip, subnet, gw);      // static — no DHCP wait
  ws.begin();
  ws.onEvent(onWsEvent);
  ws.enableHeartbeat(15000, 3000, 2);  // ping every 15s, 3s timeout, drop after 2 misses
}

void loop(){ ws.loop(); }

Verify the exact Ethernet.begin overload and enableHeartbeat signature against your
installed library versions — those APIs drift.

---

iPad client JS

Send a binary frame — [code byte][JSON] — so the Teensy reads the code without parsing.

let ws;
function connect(){
  ws = new WebSocket("ws://192.168.1.50:8080");
  ws.binaryType = "arraybuffer";
  ws.onclose = () => setTimeout(connect, 500);   // auto-reconnect
  ws.onerror = () => ws.close();
}
connect();

function sendReveal({ seq, code, trial, pred, actual, error, rt }){
  // always write your local behavioral log here, regardless of socket state
  if (!ws || ws.readyState !== WebSocket.OPEN) return;
  const body = new TextEncoder().encode(
    JSON.stringify({ seq, t_js: performance.now(), trial, pred, actual, error, rt })
  );
  const pkt = new Uint8Array(1 + body.length);
  pkt[0] = code & 0xFF;
  pkt.set(body, 1);
  ws.send(pkt.buffer);
}

---

iOS-specific gotchas

• Connection dies on sleep / backgrounding. WebKit suspends timers and closes sockets when
  the tab isn't foregrounded. Per session: disable auto-lock, keep the task tab foreground, rely
  on onclose → reconnect. The Teensy heartbeat catches half-open connections the iPad doesn't
  cleanly close.
• Keep writing the browser log unconditionally. sendReveal logs locally before checking
  socket state — a dropped connection costs a timing trigger for those trials (recoverable-ish
  via photodiode), never the metadata. Browser log stays ground truth; seq drives the merge.

---

Timing & offline merge (the methodological crux)

Three clocks never agree: browser performance.now(), Teensy micros(), DAQ clock. The
hardware trigger is the only thing tying browser events to neural time.

• DAQ records (code, neural-timestamp).
• Browser records (seq, code, full metadata, t_js).
• Match on seq offline to hang metadata on each neural-time trigger.
• Teensy also logs receipt micros() + seq — not for analysis, but to characterize
  latency/jitter and catch drops.

Caveat to state loudly: the network trigger fires when JS emits the reveal event, which is
one frame + panel latency (~16.7 ms @ 60 Hz, plus display lag) away from photons changing on
screen, and WiFi + event-loop jitter adds several ms on top. If you need ms-accurate
visual-onset-locked responses, put a photodiode on the reveal region into a DAQ analog
channel and treat that as onset truth. The network trigger is then a coarse marker + metadata
key. If you only need event identity + behavioral RT, the network trigger alone is fine.

---

DAQ output format (pin plan depends on this)

What the trigger physically is depends on your DAQ's digital input:

• Single TTL pulse on a BNC — marks "an event," no content. Simplest.
• N-bit parallel + strobe — 8 GPIO carry the code, one strobe latches it. Classic pattern
  for Blackrock / Ripple / Neuralynx digital ports; almost certainly what you want for ECoG
  since it distinguishes event types in the neural stream. (This is what the sketch assumes.)
• Serial to a trigger box, if your system takes that.

Confirm which your acquisition system expects before wiring — it determines the pin plan.

---

Bring-up sequence

1. Reachability first. From the iPad, load http://192.168.1.50:8080/ in the browser — even
   a failed/blank response confirms the iPad routes to the Teensy. If it times out, it's a
   network/subnet problem, not code, and nothing else works until fixed.
2. Handshake. Open the WS from a desktop browser console on the same LAN; confirm
   WStype_CONNECTED prints on Teensy Serial. Desktop-first isolates iOS quirks from library quirks.
3. Trigger. Scope the strobe + code pins while firing test frames; confirm width and code bits.
4. iPad end-to-end. Run the task, watch Serial for the JSON tail + seq incrementing with no gaps.
5. Timing truth. Photodiode on the reveal region into a DAQ analog channel; compare to the
   strobe across a few hundred trials to characterize WiFi lag + jitter. Keep that distribution —
   network trigger stays a coarse marker + metadata key, photodiode is onset truth.

---

Open questions to close the spec

• Does the DAQ digital input want a single TTL or 8-bit parallel + strobe?
• Does the bridging router/AP already exist in the rig, or does it need adding?