graphprotocol / graphprotocol/indexer
indexer-agent reconciliation loop overrides manual graphman pause
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 262
- Forks
- 148
- PR merge metrics
- No merged PRs in 30d
Description
Summary
When using graphman pause <deployment> to manually pause a subgraph, the indexer-agent's reconcileDeployments loop automatically resumes it within minutes by calling subgraph_deploy on the graph-node JSON-RPC admin API. This makes it impossible to keep a subgraph paused for maintenance operations such as graphman rewind.
Steps to reproduce
graphman pause <IPFS_HASH>- Wait 2-5 minutes (one reconciliation cycle)
graphman info <IPFS_HASH> --status→ showsPaused: false
The subgraph has decisionBasis: always in indexer rules.
Expected behavior
The reconciliation loop should check the paused_at field before calling subgraph_deploy. If a deployment was explicitly paused via graphman, the agent should not resume it.
Impact
- Cannot perform
graphman rewindsafely — the subgraph resumes indexing during the rewind, causing data inconsistencies - Any maintenance operation requiring a temporary stop is compromised
- Operators must resort to workarounds like reassigning to a non-existent node (e.g.
graphman reassign <hash> maintenance_node_0) to prevent the agent from resuming the subgraph
Root cause
In packages/indexer-agent/src/agent.ts, the reconcileDeployments function calculates target deployments based on indexer rules and calls this.graphNode.ensure() for each one. There is no check for whether a deployment is currently paused before calling ensure. If the deployment is in the target list (active allocation or decisionBasis: always/offchain), it will be re-deployed, which implicitly resumes it.
Suggested fix
Before calling ensure for a deployment, check if it is currently paused (paused_at IS NOT NULL). If paused, skip the ensure call. Optionally, add a --force flag to graphman pause or a new indexer rule field (e.g. maintenancePause: true) that the agent explicitly respects.
Related issues
- #713 — Added
--deployment-management manualmode as a workaround (disables entire reconciliation loop) - #853 — Aimed to make the agent respect the new pause mechanism, but the "use new pausing mechanism" task was never completed
- graphprotocol/graph-node#4255 — Introduced first-class
graphman pause/resumecommands - graphprotocol/graph-node#5253 —
graphman reassignshould pause the subgraph first (still OPEN) - graphprotocol/graph-node#5110 —
graphman rewindshould use the pause mechanism (closed by stale bot, never implemented)
Current workaround
Reassign the subgraph to a non-existent node before maintenance:
graphman pause <IPFS_HASH>
graphman reassign <IPFS_HASH> maintenance_node_0
# perform maintenance (rewind, reindex, etc.)
graphman reassign <IPFS_HASH> index_node_0
graphman resume <IPFS_HASH>
The indexer-agent cannot resume the subgraph because maintenance_node_0 does not exist.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in packages/indexer-agent/src/agent.ts at reconcileDeployments and trace how target deployments reach this.graphNode.ensure(). Check how the paused_at field is exposed and reproduce the graphman pause scenario; done means an explicitly paused deployment is not resumed during reconciliation while normal deployments still reconcile.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100