KeeperHub / KeeperHub/keeperhub
Widen the trigger observability surface: state, trace, upstream feeds, and pre-inclusion
- Dominant language
- TypeScript
- Stars
- 24
- Forks
- 93
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 253
Description
Tracking issue. The work is broken into the seven issues checklisted below; discussion of the overall direction belongs here, implementation discussion belongs on the individual issues.
## Reason: what cannot be done today
On-chain observation in KeeperHub is `eth_getLogs` and nothing else (`keeperhub-events/event-tracker/src/chains/provider-manager.ts:11`). The trigger types are Manual, Schedule, Webhook, Event, Block and Transfer (`lib/workflow/store.ts:14-21`).
That means a workflow can react to exactly two things on chain: an event a contract chose to emit, and block height. Everything else is invisible. Concretely, none of these can be expressed as a trigger today:
- A reverted transaction. A failed drain attempt against a monitored contract emits nothing.
- An internal ETH transfer via `call`, a `delegatecall` into an unlogged implementation, `SELFDESTRUCT`, or contract creation.
- A privileged function on a third-party contract whose author did not emit an event for it.
- A threshold on derived state - "health factor below 1.05", "LTV above X", "collateral ratio drifted". Most protocols do not emit an event when a derived value crosses a threshold.
Monitoring a contract we do not control limits users to whatever its author decided to log. That is a large fraction of what people actually want to monitor, and it is why "add a mempool trigger" was the original framing of this work.
## Scope: what this covers, and what it does not
This tracks widening the **observability surface** - the set of on-chain and off-chain conditions that can start a workflow. Four tiers, ordered by unlock per unit of cost:
1. State and threshold triggers - `eth_call` per block against a view function, compared to a threshold (EVM only; Solana needs its own read path)
2. Trace and internal-call triggers - `debug_traceBlockByNumber` or equivalent
3. Upstream data-source feeds - oracle reports, bridge queues, timelocks
4. Pre-inclusion observation - per chain, because it is a different protocol on each
Each tier is independently shippable and separately useful, which is why they are separate issues rather than one.
**Not in scope here:** reaction *latency*. How fast a workflow runs once a trigger fires is a separate problem with a separate cause, tracked internally. It matters to tier 4 specifically - see below - but tiers 1 to 3 are useful regardless of it.
**Also not in scope:** submission-side ordering. Reacting to an observed transaction with your own transaction, positioned relative to it, requires atomic bundle submission (`eth_sendBundle`, MEV-Share backruns). That is a different integration from observation and is not part of this work.
## Plan
Build tiers 1 to 3 first. They are chain-agnostic, reuse the block subscription the event-tracker already maintains, and are deterministic - a state comparison or a trace match either holds or does not.
Tier 4 is sequenced last, and deliberately so. Two findings from the investigation behind this issue:
**Pre-inclusion observation is not one integration.** Of the 11 mainnets in our production config (22 entries including testnets), only `eth-mainnet` has a private-mempool RPC configured, and that is submission-side only - it provides no observation capability. Base and Arbitrum have no public mempool of consequence - ordering is decided by a sequencer, so the observation points are Flashblocks and the sequencer feed respectively. Solana has no mempool at all and needs Geyser or ShredStream. These share no code, which is why tier 4 is four issues.
**For most cases where lead time is valuable, the mempool is the wrong source.** The causing signal is usually observable off-chain first. For liquidation defence the useful signal is the oracle price update that will make a position liquidatable, not the liquidator's transaction - by the time that transaction is visible you are racing a searcher and will lose. Same shape for bridge messages queued on a source chain and governance actions sitting in a timelock. Tier 3 delivers real lead time over an ordinary subscription; tier 4 is for genuinely adversarial transactions with no upstream feed, which is a narrower set than it first appears.
Tier 4 also carries a semantic cost that tiers 1 to 3 do not: a pending transaction may be replaced, dropped, or land reverted, so a trigger on one is speculative and needs distinct product and billing semantics rather than being a faster version of an existing trigger.
## Sub-issues
Contributions are welcome on tiers 1 to 3. Tier 4 issues are filed for visibility and design discussion; please read the sequencing note on each before starting work.
**Tier 1 - state and threshold triggers**
- [ ] #2240 - Workflows cannot trigger on a threshold over contract state, only on emitted events
**Tier 2 - trace and internal-call triggers**
- [ ] #2241 - Reverted transactions and internal calls are invisible to triggers: eth_getLogs sees only emitted events
- [ ] #2247 - Survey trace-method availability and pricing across configured chain upstreams (`good first issue`)
**Tier 3 - upstream data-source feeds**
- [ ] #2242 - No way to trigger on an upstream signal (oracle report, bridge queue, timelock) before it lands on chain
**Tier 4 - pre-inclusion, one issue per protocol family**
- [ ] #2243 - Pre-inclusion triggers on chains with a public mempool (eth-mainnet, BSC, Polygon, Avalanche)
- [ ] #2244 - Pre-inclusion triggers on Base need Flashblocks, not a mempool
- [ ] #2245 - Pre-inclusion triggers on Arbitrum via the sequencer feed
- [ ] #2246 - Pre-inclusion visibility on Solana: Geyser vs ShredStream spike
## One design problem that spans tiers 1 and 2
A state or trace trigger fires on a *condition*, not a discrete event, so a naive implementation fires on every block for as long as the condition holds. These triggers need edge detection and either hysteresis or an explicit re-arm rule.
The existing dedup store keys on `transactionHash` (`keeperhub-events/event-tracker/src/listener/event-listener.ts:200-220`), which does not exist for a state-derived trigger. A new dedup identity is required. This should be settled before implementation on either tier, and is the most likely thing to make two otherwise-good pull requests incompatible.
---
Tracking: [KEEP-1270](https://linear.app/keeperhubapp/issue/KEEP-1270) (internal tracker, not publicly accessible)
Contributor guide
Research direction
This is a tracking and design issue, not a standalone implementation task. Start with the relevant sub-issue, then read keeperhub-events/event-tracker/src/chains/provider-manager.ts and lib/workflow/store.ts; inspect event-listener.ts:200-220 for the existing deduplication constraint. Done means the selected sub-issue has a decided scope and implementation-ready acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- blockchain, typescript
- Domain
- backend, blockchain, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100