KeeperHub / KeeperHub/keeperhub

Widen the trigger observability surface: state, trace, upstream feeds, and pre-inclusion

Open
#2,239 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
24
Forks
93
Avg merge
1d 4h
Merged PRs (30d)
253

Description

Tracking issue. The work is broken into the seven issues checklisted below; discussion of the overall direction belongs here, implementation discussion belongs on the individual issues.

## Reason: what cannot be done today

On-chain observation in KeeperHub is `eth_getLogs` and nothing else (`keeperhub-events/event-tracker/src/chains/provider-manager.ts:11`). The trigger types are Manual, Schedule, Webhook, Event, Block and Transfer (`lib/workflow/store.ts:14-21`).

That means a workflow can react to exactly two things on chain: an event a contract chose to emit, and block height. Everything else is invisible. Concretely, none of these can be expressed as a trigger today:

- A reverted transaction. A failed drain attempt against a monitored contract emits nothing.
- An internal ETH transfer via `call`, a `delegatecall` into an unlogged implementation, `SELFDESTRUCT`, or contract creation.
- A privileged function on a third-party contract whose author did not emit an event for it.
- A threshold on derived state - "health factor below 1.05", "LTV above X", "collateral ratio drifted". Most protocols do not emit an event when a derived value crosses a threshold.

Monitoring a contract we do not control limits users to whatever its author decided to log. That is a large fraction of what people actually want to monitor, and it is why "add a mempool trigger" was the original framing of this work.

## Scope: what this covers, and what it does not

This tracks widening the **observability surface** - the set of on-chain and off-chain conditions that can start a workflow. Four tiers, ordered by unlock per unit of cost:

1. State and threshold triggers - `eth_call` per block against a view function, compared to a threshold (EVM only; Solana needs its own read path)
2. Trace and internal-call triggers - `debug_traceBlockByNumber` or equivalent
3. Upstream data-source feeds - oracle reports, bridge queues, timelocks
4. Pre-inclusion observation - per chain, because it is a different protocol on each

Each tier is independently shippable and separately useful, which is why they are separate issues rather than one.

**Not in scope here:** reaction *latency*. How fast a workflow runs once a trigger fires is a separate problem with a separate cause, tracked internally. It matters to tier 4 specifically - see below - but tiers 1 to 3 are useful regardless of it.

**Also not in scope:** submission-side ordering. Reacting to an observed transaction with your own transaction, positioned relative to it, requires atomic bundle submission (`eth_sendBundle`, MEV-Share backruns). That is a different integration from observation and is not part of this work.

## Plan

Build tiers 1 to 3 first. They are chain-agnostic, reuse the block subscription the event-tracker already maintains, and are deterministic - a state comparison or a trace match either holds or does not.

Tier 4 is sequenced last, and deliberately so. Two findings from the investigation behind this issue:

**Pre-inclusion observation is not one integration.** Of the 11 mainnets in our production config (22 entries including testnets), only `eth-mainnet` has a private-mempool RPC configured, and that is submission-side only - it provides no observation capability. Base and Arbitrum have no public mempool of consequence - ordering is decided by a sequencer, so the observation points are Flashblocks and the sequencer feed respectively. Solana has no mempool at all and needs Geyser or ShredStream. These share no code, which is why tier 4 is four issues.

**For most cases where lead time is valuable, the mempool is the wrong source.** The causing signal is usually observable off-chain first. For liquidation defence the useful signal is the oracle price update that will make a position liquidatable, not the liquidator's transaction - by the time that transaction is visible you are racing a searcher and will lose. Same shape for bridge messages queued on a source chain and governance actions sitting in a timelock. Tier 3 delivers real lead time over an ordinary subscription; tier 4 is for genuinely adversarial transactions with no upstream feed, which is a narrower set than it first appears.

Tier 4 also carries a semantic cost that tiers 1 to 3 do not: a pending transaction may be replaced, dropped, or land reverted, so a trigger on one is speculative and needs distinct product and billing semantics rather than being a faster version of an existing trigger.

## Sub-issues

Contributions are welcome on tiers 1 to 3. Tier 4 issues are filed for visibility and design discussion; please read the sequencing note on each before starting work.

**Tier 1 - state and threshold triggers**
- [ ] #2240 - Workflows cannot trigger on a threshold over contract state, only on emitted events

**Tier 2 - trace and internal-call triggers**
- [ ] #2241 - Reverted transactions and internal calls are invisible to triggers: eth_getLogs sees only emitted events
- [ ] #2247 - Survey trace-method availability and pricing across configured chain upstreams (`good first issue`)

**Tier 3 - upstream data-source feeds**
- [ ] #2242 - No way to trigger on an upstream signal (oracle report, bridge queue, timelock) before it lands on chain

**Tier 4 - pre-inclusion, one issue per protocol family**
- [ ] #2243 - Pre-inclusion triggers on chains with a public mempool (eth-mainnet, BSC, Polygon, Avalanche)
- [ ] #2244 - Pre-inclusion triggers on Base need Flashblocks, not a mempool
- [ ] #2245 - Pre-inclusion triggers on Arbitrum via the sequencer feed
- [ ] #2246 - Pre-inclusion visibility on Solana: Geyser vs ShredStream spike

## One design problem that spans tiers 1 and 2

A state or trace trigger fires on a *condition*, not a discrete event, so a naive implementation fires on every block for as long as the condition holds. These triggers need edge detection and either hysteresis or an explicit re-arm rule.

The existing dedup store keys on `transactionHash` (`keeperhub-events/event-tracker/src/listener/event-listener.ts:200-220`), which does not exist for a state-derived trigger. A new dedup identity is required. This should be settled before implementation on either tier, and is the most likely thing to make two otherwise-good pull requests incompatible.

---
Tracking: [KEEP-1270](https://linear.app/keeperhubapp/issue/KEEP-1270) (internal tracker, not publicly accessible)

Contributor guide

Open the contributing guide

Research direction

This is a tracking and design issue, not a standalone implementation task. Start with the relevant sub-issue, then read keeperhub-events/event-tracker/src/chains/provider-manager.ts and lib/workflow/store.ts; inspect event-listener.ts:200-220 for the existing deduplication constraint. Done means the selected sub-issue has a decided scope and implementation-ready acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
blockchain, typescript
Domain
backend, blockchain, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.