opensearch-project / opensearch-project/opensearch-java

[BUG] `settings.index.knn.algo_param.ef_search` skipped during deserialization

Open
#1,414 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
165
Forks
250
Avg merge
1d 18h
Merged PRs (30d)
26

Description

What is the bug?

A clear and concise description of the bug.

The settings.index.knn.algo_param.ef_search field does not appear to be deserialized correctly.

How can one reproduce the bug?

Steps to reproduce the behavior.

  1. Create an index with settings.index.knn.algo_param.ef_search
CreateIndexRequest.Builder createIndexRequestBuilder = new CreateIndexRequest.Builder().index("bug-repro-index");
IndexSettings.Builder indexSettingsBuilder = new IndexSettings.Builder();
indexSettingsBuilder.knn(true);
indexSettingsBuilder.knnAlgoParamEfSearch(512);
CreateIndexResponse createIndexResponse = client.indices().create(createIndexRequestBuilder.settings(indexSettingsBuilder.build()).build());
  1. Get the index
GetIndexRequest getIndexRequest = new GetIndexRequest.Builder().index("bug-repro-index").build();
GetIndexResponse getIndexResponse = client.indices().get(getIndexRequest);
if (getIndexResponse.result().get("bug-repro-index").settings().index().knnAlgoParamEfSearch() == null) {
    System.out.println("This is a bug");
}
  1. See that the field is null
What is the expected behavior?

A clear and concise description of what you expected to happen.

The settings.index.knn.algo_param.ef_search field should be not null and should be set to the expected value of 512 in the example.

What is your host/environment?

Operating system, version.

Do you have any screenshots?

If applicable, add screenshots to help explain your problem.

N/A

Do you have any additional context?

Add any other context about the problem.

When examining the response from GET /bug-repro-index from the OpenSearch Dashboard, it is shaped like

{
  "bug-repro-index": {
    "aliases": {},
    "mappings": {},
    "settings": {
      "index": {
        "number_of_shards": "2",
        "knn": {
          "algo_param": {
            "ef_search": "512"
          }
        },
        "provided_name": "bug-repro-index",
        "knn": "true",
        "creation_date": "1738883254271",
        "number_of_replicas": "0",
        "uuid": "K2qJ1UPLcQdi9m5AxzXr",
        "version": {
          "created": "136327827"
        }
      }
    }
  }
}

Notice that the response has settings.index.knn.algo_param.ef_search as a nested object rather than a flattened field. After debugging and stepping through the code, seems like there are a few contributing issues:

  1. There is a map of field names to deserializers (see code). When the response is being deserialized, the field name is knn.algo_param. However, the map does not contain knn.algo_param - it only contains knn.algo_param.ef_search and index.knn.algo_param.ef_search. Because this field name is not recognized, it gets skipped (see code).
  2. It seems like the generated code is expecting an integer for deserialization. However, I'd expect it to use a deserializer for an object, similar to the settings.index.version field (see generated code).

Minimal Java class used for testing

package org.example;

import org.opensearch.client.opensearch.OpenSearchClient;
import org.opensearch.client.opensearch.indices.CreateIndexRequest;
import org.opensearch.client.opensearch.indices.CreateIndexResponse;
import org.opensearch.client.opensearch.indices.GetIndexRequest;
import org.opensearch.client.opensearch.indices.GetIndexResponse;
import org.opensearch.client.opensearch.indices.IndexSettings;
import org.opensearch.client.transport.aws.AwsSdk2Transport;
import org.opensearch.client.transport.aws.AwsSdk2TransportOptions;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.http.SdkHttpClient;
import software.amazon.awssdk.http.apache.ApacheHttpClient;
import software.amazon.awssdk.regions.Region;

import java.io.IOException;

public class Main {
    public static void main(String[] args) throws IOException {
        SdkHttpClient httpClient = ApacheHttpClient.builder().build();
        AwsCredentialsProvider provider = DefaultCredentialsProvider.create();

        OpenSearchClient client = new OpenSearchClient(
            new AwsSdk2Transport(
                httpClient,
                "collectionid.us-east-1.aoss.amazonaws.com", // OpenSearch endpoint, without https://
                "aoss",
                Region.US_EAST_1,
                AwsSdk2TransportOptions.builder().setCredentials(provider).build()
            )
        );

        CreateIndexRequest.Builder createIndexRequestBuilder = new CreateIndexRequest.Builder().index("bug-repro-index");
        IndexSettings.Builder indexSettingsBuilder = new IndexSettings.Builder();
        indexSettingsBuilder.knn(true);
        indexSettingsBuilder.knnAlgoParamEfSearch(512);
        CreateIndexResponse createIndexResponse = client.indices().create(
            createIndexRequestBuilder.settings(indexSettingsBuilder.build()).build());


        GetIndexRequest getIndexRequest = new GetIndexRequest.Builder().index("bug-repro-index").build();
        GetIndexResponse getIndexResponse = client.indices().get(getIndexRequest);
        if (getIndexResponse.result().get("bug-repro-index").settings().index().knnAlgoParamEfSearch() == null) {
            System.out.println("This is a bug");
        }

        httpClient.close();
    }
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ObjectDeserializer.java, especially the field-name deserializer map and the skipped-field path, then compare the generated IndexSettings.java handling for knnAlgoParamEfSearch with the index.version object handling. Reproduce the issue with the Java example and verify that GET index deserializes settings.index.knn.algo_param.ef_search as 512 rather than null.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.