elastic / elastic/elasticsearch-java

Intermittent `Unexpected token [START_OBJECT] found after the main object` error when compression is enabled

Open
#1,165 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
524
Forks
300
Avg merge
1d 11h
Merged PRs (30d)
16

Description

### Java API client version

9.2.4

### Java version

25

### Elasticsearch Version

9.2.4

### Problem description

### Precondition

Client is initiated with `compressionEnabled = true`

```
var rest5Client = Rest5Client.builder(uri)
.setCompressionEnabled(true)
.build();
var transport = new Rest5ClientTransport(rest5Client, new JacksonJsonpMapper());

elasticClient = new ElasticsearchClient(transport);

elasticClient.search(validSearchRequestOf599Length);
```

### Problem

For valid search request the client sometimes returns 400 Bad Request:

```
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "Unexpected token [START_OBJECT] found after the main object.",
"line": 1,
"col": 600
}
],
"type": "parsing_exception",
"reason": "Unexpected token [START_OBJECT] found after the main object.",
"line": 1,
"col": 600
},
"status": 400
}
```

⚠️ In this example the initial request length is 599, but error states that at col 600 there is `{` symbol in the request.

### Root cause: `MultiBufferEntity` in `elasticsearch-java` 9.2.4

The `Rest5ClientHttpClient` (in `elasticsearch-java` 9.2.4) wraps the serialized request body in a `MultiBufferEntity` — an `AsyncEntityProducer` used by Apache HttpClient 5's async pipeline.

Key characteristics of `MultiBufferEntity`:

| Property | Value | Implication |
|-----------------------|--------|--------------------------------------------------------------------------|
| `isRepeatable()` | `true` | Tells Apache HC5 the entity can be replayed |
| `getContentLength()` | `-1` | Chunked transfer encoding — no fixed content-length to guard body length |
| `releaseResources()` | resets iterator | Called by the async pipeline, resets to beginning of buffers |

With `getContentLength() = -1`, Apache HC5 uses **chunked transfer encoding**. The `MultiBufferEntity` implements both the synchronous `writeTo(OutputStream)` and asynchronous `produce(DataStreamChannel)` methods independently. In the async pipeline, the interaction between these two code paths, the entity's state management, and Apache HC5's async execution chain can result in the body content being **produced twice** into the outgoing stream:

1. The `produce()` calls write the full 599-byte body to the `DataStreamChannel`.
2. Under certain async pipeline conditions, the entity's state is reset via `releaseResources()` → `init()`.
3. The `produce()` calls resume and write the body **again**, appending to what was already sent.
4. Elasticsearch receives 1198 bytes: the original body followed by a duplicate — and rejects it at col 600.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.