elastic / elastic/elasticsearch-java
Intermittent `Unexpected token [START_OBJECT] found after the main object` error when compression is enabled
- Dominant language
- Java
- Stars
- 524
- Forks
- 300
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 16
Description
### Java API client version
9.2.4
### Java version
25
### Elasticsearch Version
9.2.4
### Problem description
### Precondition
Client is initiated with `compressionEnabled = true`
```
var rest5Client = Rest5Client.builder(uri)
.setCompressionEnabled(true)
.build();
var transport = new Rest5ClientTransport(rest5Client, new JacksonJsonpMapper());
elasticClient = new ElasticsearchClient(transport);
elasticClient.search(validSearchRequestOf599Length);
```
### Problem
For valid search request the client sometimes returns 400 Bad Request:
```
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "Unexpected token [START_OBJECT] found after the main object.",
"line": 1,
"col": 600
}
],
"type": "parsing_exception",
"reason": "Unexpected token [START_OBJECT] found after the main object.",
"line": 1,
"col": 600
},
"status": 400
}
```
⚠️ In this example the initial request length is 599, but error states that at col 600 there is `{` symbol in the request.
### Root cause: `MultiBufferEntity` in `elasticsearch-java` 9.2.4
The `Rest5ClientHttpClient` (in `elasticsearch-java` 9.2.4) wraps the serialized request body in a `MultiBufferEntity` — an `AsyncEntityProducer` used by Apache HttpClient 5's async pipeline.
Key characteristics of `MultiBufferEntity`:
| Property | Value | Implication |
|-----------------------|--------|--------------------------------------------------------------------------|
| `isRepeatable()` | `true` | Tells Apache HC5 the entity can be replayed |
| `getContentLength()` | `-1` | Chunked transfer encoding — no fixed content-length to guard body length |
| `releaseResources()` | resets iterator | Called by the async pipeline, resets to beginning of buffers |
With `getContentLength() = -1`, Apache HC5 uses **chunked transfer encoding**. The `MultiBufferEntity` implements both the synchronous `writeTo(OutputStream)` and asynchronous `produce(DataStreamChannel)` methods independently. In the async pipeline, the interaction between these two code paths, the entity's state management, and Apache HC5's async execution chain can result in the body content being **produced twice** into the outgoing stream:
1. The `produce()` calls write the full 599-byte body to the `DataStreamChannel`.
2. Under certain async pipeline conditions, the entity's state is reset via `releaseResources()` → `init()`.
3. The `produce()` calls resume and write the body **again**, appending to what was already sent.
4. Elasticsearch receives 1198 bytes: the original body followed by a duplicate — and rejects it at col 600.
Contributor guide
Assessment
This issue has not been assessed yet.