JanusGraph / JanusGraph/janusgraph
ES bulk request entity too large while executing index operation
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
- Version: 0.5.1
- Storage Backend: HBase
- Mixed Index Backend: ElasticSearch 6.1.3
- Mailing list Thread URL:
- Steps to Reproduce: Executing a reindex job on mapreduce
### Stack Trace
```
2020-06-04 18:50:30,817 282383 [Thread-19] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local112793509_0001
java.lang.Exception: org.janusgraph.core.JanusGraphException: Unknown exception while executing index operation
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: org.janusgraph.core.JanusGraphException: Unknown exception while executing index operation
at org.janusgraph.graphdb.olap.job.IndexRepairJob.workerIterationEnd(IndexRepairJob.java:206)
at org.janusgraph.graphdb.olap.VertexJobConverter.workerIterationEnd(VertexJobConverter.java:118)
at org.janusgraph.hadoop.scan.HadoopScanMapper.cleanup(HadoopScanMapper.java:130)
at org.janusgraph.hadoop.scan.HadoopVertexScanMapper.cleanup(HadoopVertexScanMapper.java:55)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:149)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.janusgraph.diskstorage.PermanentBackendException: Unknown exception while executing index operation
at org.janusgraph.diskstorage.es.ElasticSearchIndex.convert(ElasticSearchIndex.java:447)
at org.janusgraph.diskstorage.es.ElasticSearchIndex.restore(ElasticSearchIndex.java:895)
at org.janusgraph.diskstorage.indexing.IndexTransaction.restore(IndexTransaction.java:128)
at org.janusgraph.graphdb.olap.job.IndexRepairJob.workerIterationEnd(IndexRepairJob.java:201)
... 12 more
Caused by: org.elasticsearch.client.ResponseException: method [POST], host [http://10.1.***.***:9200], URI [/_bulk], status line [HTTP/1.1 413 Request Entity Too Large]
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:283)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:261)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.performRequest(RestElasticSearchClient.java:483)
at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.performRequest(RestElasticSearchClient.java:474)
at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.bulkRequest(RestElasticSearchClient.java:397)
at org.janusgraph.diskstorage.es.ElasticSearchIndex.restore(ElasticSearchIndex.java:893)
... 14 more
```
Contributor guide
Research direction
Start with ElasticSearchIndex.restore and convert, then follow the call into RestElasticSearchClient.bulkRequest shown in the stack trace. Reproduce the mapreduce reindex job against Elasticsearch 6.1.3 and inspect the bulk request failure; done means the index operation no longer fails with HTTP 413 Request Entity Too Large.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, java
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100