graphprotocol / graphprotocol/indexer
indexer-agent v0.25.10: TAPv2 GraphTallyCollector runs one cycle after boot, then silently never again
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 262
- Forks
- 148
- PR merge metrics
- No merged PRs in 30d
Description
Summary
indexer-agent v0.25.10's TAPv2 collector (GraphTallyCollector) runs exactly one redemption cycle after process start and then never again, while the rest of the agent (reconcile loops, NetworkMonitor, actions queue) keeps running normally. The process looks fully healthy — pod Ready, thousands of log lines per 10 minutes — but no [TAPv2] log line appears for 6+ hours after the initial cycle. We hit this twice in three days, and it delayed redemption of ~42k GRT until a manual restart.
Environment
ghcr.io/graphprotocol/indexer-agent:v0.25.10- Arbitrum One, Horizon enabled (
INDEXER_HORIZON__ENABLED=trueon the paired indexer-rs stack) - Indexer
0xEdca8740873152fF30a2696Add66D1ab41882beB(also observed cycle-once behaviour on a second identity,0x3717CE…, same version)
Observed timeline (occurrence 2, 2026-08-30 UTC)
rollout restart→ agent boots, and ~2 minutes later runs a full TAPv2 cycle:Query RAVs eligible for redemption→RAVs passed escrow balance check→Submit RAVs on chain via batched multicall→ (at that time each collect reverted for an unrelated on-chain reason).- For the following 6 hours: zero
[TAPv2]lines. Meanwhile the same window shows ~1,400 log lines/10 min of normal Agent/NetworkMonitor activity. 323 RAVs remained eligible the whole time (above threshold, allocations closed). - Second
rollout restart→ TAPv2 cycle fires within 2 minutes →Batch redemption completed, 323 RAVs redeemed (~42k GRT).
Occurrence 1 (2026-08-28) was the harder variant: the agent booted while its Postgres was briefly unreachable, logged Unhandled promise rejection at startup, and then produced no log lines at all for 2 days while the pod stayed Ready (liveness only checks the HTTP port).
Expected
The RAV redemption loop runs on its interval for the lifetime of the process, or the process exits/fails visibly when the loop dies.
Suspicion
The one-cycle-then-silence pattern suggests the collector's loop promise rejects/errors after the first iteration (e.g., when a batch fails validation) without being rescheduled or surfacing — consistent with the startup Unhandled promise rejection variant being fatal to individual loops but not the process.
Ask
- Reschedule/restart the TAPv2 loop on error, or crash the process so orchestration restarts it.
- A heartbeat log line (or metric) for the collector loop so operators can alert on its absence.
Happy to provide fuller logs privately.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the TAPv2 GraphTallyCollector entry point and trace the redemption loop promise through its first cycle and error paths. Reproduce the one-cycle-then-silence behavior using the reported logs and verify that the loop either continues for the process lifetime or that its failure becomes visible and causes a restart.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100