graphprotocol / graphprotocol/indexer
[ISSUE] status endpoint crash
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 262
- Forks
- 148
- PR merge metrics
- No merged PRs in 30d
Description
If I make a larger query like this:
graph indexer allocations get --deployment QmYNPk3Mabrx4VmxPKmBevdi11cfVUfTXVtgygG4M4m9tQ
where the SG allocation is very old or the query itself is very abstract and the API needs to perform a larger query, I notice that the status API endpoint is crashing with no logs related.
This command above produces this error:
✖ [GraphQL] Failed to query Epoch Block Oracle Subgraph
then I query the status network
graph indexer status network --network arbitrum-one
and I get this output with an error indicating that (possibly) the JVM crashed (again no logs and no pods restarted)
graph@indexer-cli:~$ graph indexer status network --network arbitrum-one
Registration
┌───────────────────────────────────────┬────────────────────────────────────────────┬─────────────────┬────────────┬─────────────────────────────────────┐
│ url │ address │ protocolNetwork │ registered │ location │
├───────────────────────────────────────┼────────────────────────────────────────────┼─────────────────┼────────────┼─────────────────────────────────────┤
│ https://indexer.upgrade.thegraph.com/ │ 0xbdfb5Ee5a2aBF4fc7bB1bD1221067aEf7f9dE491 │ eip155:42161 │ true │ 89.99997854232788,36.57848596572876 │
└───────────────────────────────────────┴────────────────────────────────────────────┴─────────────────┴────────────┴─────────────────────────────────────┘
Endpoints
┌─────────┬─────────────────┬─────────────────────────────────────────────┬────────┐
│ name │ protocolNetwork │ url │ status │
├─────────┼─────────────────┼─────────────────────────────────────────────┼────────┤
│ service │ arbitrum-one │ https://indexer.upgrade.thegraph.com/ │ down │
├─────────┼─────────────────┼─────────────────────────────────────────────┼────────┤
│ status │ arbitrum-one │ https://indexer.upgrade.thegraph.com/status │ up │
└─────────┴─────────────────┴─────────────────────────────────────────────┴────────┘
The following endpoint tests failed:
https://indexer.upgrade.thegraph.com/
Test: http get https://indexer.upgrade.thegraph.com/
Error: fetch failed
Possible actions:
- Make sure https://indexer.upgrade.thegraph.com/ can be resolved and reached from this machine
- Make sure the port of https://indexer.upgrade.thegraph.com/ is set up correctly
- Make sure the test command returns an HTTP status code < 400
Indexer Deployments
<--- Last few GCs --->
[26273:0x64f1df0] 83686 ms: Mark-Compact (reduce) 4090.9 (4116.2) -> 4090.9 (4109.2) MB, 35.44 / 0.00 ms (+ 8.5 ms in 1 steps since start of marking, biggest step 8.5 ms, walltime since start of marking 48 ms) (average mu = 0.501, current mu = 0.311) [26273:0x64f1df0] 83747 ms: Mark-Compact (reduce) 4093.2 (4111.5) -> 4093.2 (4111.5) MB, 55.82 / 0.00 ms (average mu = 0.340, current mu = 0.078) allocation failure; scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
1: 0xca5580 node::Abort() [node]
2: 0xb781f9 [node]
3: 0xeca4d0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
4: 0xeca7b7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
5: 0x10dc505 [node]
6: 0x10f4388 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
7: 0x10ca4a1 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
8: 0x10cb635 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
9: 0x10a8c86 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
10: 0x1503a16 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
11: 0x7daacfed9ef6
Aborted
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with graph indexer allocations get --deployment QmYNPk3Mabrx4VmxPKmBevdi11cfVUfTXVtgygG4M4m9tQ, then query graph indexer status network --network arbitrum-one and inspect the status endpoint path involved. Trace the large query's memory use and add coverage if an existing test location is identified; done means the endpoint stays available without a Node.js heap-out-of-memory crash.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- graphql, node.js, typescript
- Domain
- api, backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100