JanusGraph / JanusGraph/janusgraph

Timeout exception in direct index query.

Open
#711 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
5.8k
Forks
1.2k
Avg merge
13h 53m
Merged PRs (30d)
6

Description

Sometimes direct index query throws `TimeoutException`.
There is also similar problem with detailed explanation https://github.com/elastic/elasticsearch/issues/24069

```
Caused by: org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57)
at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470)
at org.janusgraph.diskstorage.BackendTransaction.totals(BackendTransaction.java:464)
at org.janusgraph.graphdb.database.IndexSerializer.executeTotals(IndexSerializer.java:663)
at org.janusgraph.graphdb.query.graph.IndexQueryBuilder.executeTotals(IndexQueryBuilder.java:199)
at org.janusgraph.graphdb.query.graph.IndexQueryBuilder.edgeTotals(IndexQueryBuilder.java:244)
...
... 8 more
Caused by: org.janusgraph.diskstorage.PermanentBackendException: Permanent exception while executing backend operation Totals
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:82)
at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55)
... 22 more
Caused by: java.lang.RuntimeException: error while performing request
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:681)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:219)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:191)
at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.performRequest(RestElasticSearchClient.java:320)
at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.search(RestElasticSearchClient.java:276)
at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.search(RestElasticSearchClient.java:53)
at org.janusgraph.diskstorage.es.ElasticSearchIndex.runCommonQuery(ElasticSearchIndex.java:974)
at org.janusgraph.diskstorage.es.ElasticSearchIndex.totals(ElasticSearchIndex.java:993)
at org.janusgraph.diskstorage.indexing.IndexTransaction.totals(IndexTransaction.java:129)
at org.janusgraph.diskstorage.BackendTransaction$TotalsCallable.call(BackendTransaction.java:453)
at org.janusgraph.diskstorage.BackendTransaction$TotalsCallable.call(BackendTransaction.java:442)
at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69)
... 23 more
Caused by: java.util.concurrent.TimeoutException
at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequest(AbstractNIOConnPool.java:364)
at org.apache.http.nio.pool.AbstractNIOConnPool.processNextPendingRequest(AbstractNIOConnPool.java:344)
at org.apache.http.nio.pool.AbstractNIOConnPool.release(AbstractNIOConnPool.java:318)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.releaseConnection(PoolingNHttpClientConnectionManager.java:303)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.releaseConnection(AbstractClientExchangeHandler.java:239)
at org.apache.http.impl.nio.client.MainClientExec.responseCompleted(MainClientExec.java:387)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:168)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
at java.lang.Thread.run(Thread.java:748)
```

Contributor guide

Open the contributing guide

Research direction

Start with the stack trace at IndexQueryBuilder.executeTotals and follow the call path through IndexSerializer, BackendTransaction, and ElasticSearchIndex.runCommonQuery. Compare the behavior with the linked Elasticsearch issue and investigate the RestElasticSearchClient request path. Done means the direct index query no longer intermittently fails with the reported TimeoutException, with regression coverage or a confirmed reproduction result.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, java
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.