sip-protocol / sip-protocol/sipher

chore(scheduled-ops): COURIER hardening (retries, observability, single-flight)

Open
#285 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
TypeScript
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Background

Spec 5 PR-C. Sipher's COURIER crank (`packages/agent/src/crank.ts:99-113`) is coarse — works for happy-path scheduled ops but isn't production-hardened. Lands after PR-A and PR-B unblock the actual broadcast paths.

Issues to fix

  1. No retries on transient failures — network blip during broadcast leaves op as `pending`; next tick retries immediately with no backoff. Could DOS the RPC during outages.
  2. No deduplication — concurrent ticks could both pick up the same op. Solana dedupes by signature on-chain, but wasteful + race-prone.
  3. Limited observability — single line per tick. No per-op latency, no growth-hook emission correlation, no failure-rate dashboard signal.
  4. Race conditions on op_status — two cranks in parallel (e.g., multi-process deploy) could both pick up the same op. Status update isn't atomic-CAS.

What to do

  • Exponential backoff per op: `next_exec = now + min(max_backoff, base * 2^attempts)`
  • Single-flight per op: lock by op-id in a Map for the duration of a tick
  • Structured logging: emit per-op start/end events with latency + result
  • Atomic status CAS: SQLite `UPDATE ... WHERE status='pending'` returning rowcount, only proceed if rowcount=1
  • Metrics endpoint: `GET /admin/api/courier/stats` — last-N-tick summary

Why

  • Production-grade reliability for unattended operation
  • Prevents RPC quota exhaustion during outages
  • Operator visibility into crank behavior

Cost

~1-2 weeks (one PR). Depends on PR-A + PR-B for full validation surface (no scheduled ops actually broadcasting today).

References

  • Spec: `docs/superpowers/specs/2026-05-15-scheduled-op-broadcasts-design.md` (COURIER hardening section, lines ~212-232)
  • Predecessor: #281, blocks PR-A (#283) and PR-B integration tests

Acceptance

  • Exponential backoff verified via unit test (consecutive failures increase tick delay)
  • Single-flight lock verified — concurrent tick simulation produces one broadcast
  • `/admin/api/courier/stats` returns sane summary (per-op latency, success/fail counts, last tick timestamps)
  • No regression in existing scheduled-op creation tests

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with packages/agent/src/crank.ts:99-113 and the COURIER hardening section in docs/superpowers/specs/2026-05-15-scheduled-op-broadcasts-design.md, then inspect the existing scheduled-op creation tests and SQLite status handling. Validate the work against the listed acceptance checks: backoff, single-flight behavior, structured per-op results, and GET /admin/api/courier/stats without regressions.

Written by the indexing model from the issue text.

Assessment

Tech stack
sqlite, typescript
Domain
api, backend, databases, observability
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.