opensearch-project / opensearch-project/opensearch-java
[BUG] `settings.index.knn.algo_param.ef_search` skipped during deserialization
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 165
- Forks
- 250
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 26
Description
What is the bug?
A clear and concise description of the bug.
The settings.index.knn.algo_param.ef_search field does not appear to be deserialized correctly.
How can one reproduce the bug?
Steps to reproduce the behavior.
- Create an index with
settings.index.knn.algo_param.ef_search
CreateIndexRequest.Builder createIndexRequestBuilder = new CreateIndexRequest.Builder().index("bug-repro-index");
IndexSettings.Builder indexSettingsBuilder = new IndexSettings.Builder();
indexSettingsBuilder.knn(true);
indexSettingsBuilder.knnAlgoParamEfSearch(512);
CreateIndexResponse createIndexResponse = client.indices().create(createIndexRequestBuilder.settings(indexSettingsBuilder.build()).build());
- Get the index
GetIndexRequest getIndexRequest = new GetIndexRequest.Builder().index("bug-repro-index").build();
GetIndexResponse getIndexResponse = client.indices().get(getIndexRequest);
if (getIndexResponse.result().get("bug-repro-index").settings().index().knnAlgoParamEfSearch() == null) {
System.out.println("This is a bug");
}
- See that the field is
null
What is the expected behavior?
A clear and concise description of what you expected to happen.
The settings.index.knn.algo_param.ef_search field should be not null and should be set to the expected value of 512 in the example.
What is your host/environment?
Operating system, version.
- Used an AWS OpenSearch Serverless collection which runs OpenSearch version 2.0.x.
- Used the OpenSearch Java Client v2.20.0
Do you have any screenshots?
If applicable, add screenshots to help explain your problem.
N/A
Do you have any additional context?
Add any other context about the problem.
When examining the response from GET /bug-repro-index from the OpenSearch Dashboard, it is shaped like
{
"bug-repro-index": {
"aliases": {},
"mappings": {},
"settings": {
"index": {
"number_of_shards": "2",
"knn": {
"algo_param": {
"ef_search": "512"
}
},
"provided_name": "bug-repro-index",
"knn": "true",
"creation_date": "1738883254271",
"number_of_replicas": "0",
"uuid": "K2qJ1UPLcQdi9m5AxzXr",
"version": {
"created": "136327827"
}
}
}
}
}
Notice that the response has settings.index.knn.algo_param.ef_search as a nested object rather than a flattened field. After debugging and stepping through the code, seems like there are a few contributing issues:
- There is a map of field names to deserializers (see code). When the response is being deserialized, the field name is
knn.algo_param. However, the map does not containknn.algo_param- it only containsknn.algo_param.ef_searchandindex.knn.algo_param.ef_search. Because this field name is not recognized, it gets skipped (see code). - It seems like the generated code is expecting an integer for deserialization. However, I'd expect it to use a deserializer for an object, similar to the
settings.index.versionfield (see generated code).
Minimal Java class used for testing
package org.example;
import org.opensearch.client.opensearch.OpenSearchClient;
import org.opensearch.client.opensearch.indices.CreateIndexRequest;
import org.opensearch.client.opensearch.indices.CreateIndexResponse;
import org.opensearch.client.opensearch.indices.GetIndexRequest;
import org.opensearch.client.opensearch.indices.GetIndexResponse;
import org.opensearch.client.opensearch.indices.IndexSettings;
import org.opensearch.client.transport.aws.AwsSdk2Transport;
import org.opensearch.client.transport.aws.AwsSdk2TransportOptions;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.http.SdkHttpClient;
import software.amazon.awssdk.http.apache.ApacheHttpClient;
import software.amazon.awssdk.regions.Region;
import java.io.IOException;
public class Main {
public static void main(String[] args) throws IOException {
SdkHttpClient httpClient = ApacheHttpClient.builder().build();
AwsCredentialsProvider provider = DefaultCredentialsProvider.create();
OpenSearchClient client = new OpenSearchClient(
new AwsSdk2Transport(
httpClient,
"collectionid.us-east-1.aoss.amazonaws.com", // OpenSearch endpoint, without https://
"aoss",
Region.US_EAST_1,
AwsSdk2TransportOptions.builder().setCredentials(provider).build()
)
);
CreateIndexRequest.Builder createIndexRequestBuilder = new CreateIndexRequest.Builder().index("bug-repro-index");
IndexSettings.Builder indexSettingsBuilder = new IndexSettings.Builder();
indexSettingsBuilder.knn(true);
indexSettingsBuilder.knnAlgoParamEfSearch(512);
CreateIndexResponse createIndexResponse = client.indices().create(
createIndexRequestBuilder.settings(indexSettingsBuilder.build()).build());
GetIndexRequest getIndexRequest = new GetIndexRequest.Builder().index("bug-repro-index").build();
GetIndexResponse getIndexResponse = client.indices().get(getIndexRequest);
if (getIndexResponse.result().get("bug-repro-index").settings().index().knnAlgoParamEfSearch() == null) {
System.out.println("This is a bug");
}
httpClient.close();
}
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ObjectDeserializer.java, especially the field-name deserializer map and the skipped-field path, then compare the generated IndexSettings.java handling for knnAlgoParamEfSearch with the index.version object handling. Reproduce the issue with the Java example and verify that GET index deserializes settings.index.knn.algo_param.ef_search as 512 rather than null.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100