opensearch-project / opensearch-project/opensearch-java

[BUG] Getting 0 results with filter_path fails to deserialize

Open
#1,729 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
165
Forks
250
Avg merge
1d 18h
Merged PRs (30d)
26

Description

What is the bug?

Not 100% sure this is a bug or a missing feature, or if theres an known workaround, but when filter_path is provided on /_search to omit any fields in the _source, an empty query result will fail to deserialize with org.opensearch.client.util.MissingRequiredPropertyException: Missing required property 'HitsMetadata.hits' because the hits.hits array is missing from the response

How can one reproduce the bug?

Make a query to {indexName}/_search?filter_path=*,-hits.hits._source.someField to only exclude someField on the _source doc, with a query body that results in no hits. The client is constructed as:

new OpenSearchClient(ApacheHttpClient5TransportBuilder
                .builder(HttpHost.create(endpoint))
                .build())

Presumably using the default JSON mapper (JacksonJsonpMapper)

The opensearch response I'm getting is:

{
  "took": 6,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 0,
      "relation": "eq"
    },
    "max_score": null
  }
}

Which fails to deserialize with org.opensearch.client.util.MissingRequiredPropertyException: Missing required property 'HitsMetadata.hits'

What is the expected behavior?

Ideally the missing array property would be deserialized into an empty array, is there a workaround for this?

What is your host/environment?

Client version: org.opensearch.client:opensearch-java:3.2.0, on Java 21

Opensearch is running in a docker container: opensearchproject/opensearch:3.2.0

Do you have any screenshots?
Do you have any additional context?

This is the calling code (in Kotlin) - the query is contructed as a string, but there's no issue deserializing when there are some results

    val jsonpMapper = client._transport().jsonpMapper()

    val searchResponseDeserializer = SearchResponse.createSearchResponseDeserializer(
        JsonpDeserializer.of(MySearchDocument::class.java)
    )
    
    val request = Requests.builder()
        .endpoint("/$indexName/_search?typed_keys=true&filter_path=-hits.hits._source.omittedField")
        .method("POST")
        .json(finalQuery)
        .build()

    val responseBody: Body? = client.generic().execute(request).body.getOrNull()

    try {
        val searchResponse: SearchResponse<MySearchDocument> = responseBody
            ?.let { b -> Bodies.json(b, searchResponseDeserializer, jsonpMapper) }
            ?: throw RuntimeException("Failed to parse search response")
        return searchResponse
    } catch (e: RuntimeException) {
        logger.error(e) { "OpenSearch: Failed to search: ${responseBody?.bodyAsString()}" }
        throw e
    }

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with SearchResponse.createSearchResponseDeserializer and the HitsMetadata.hits property involved in the reported exception. Reproduce the empty-result search using filter_path, then verify that the response with no hits deserializes successfully and preserves the expected empty result behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.