opensearch-project / opensearch-project/opensearch-java

[BUG] searchShards fails on Managed OpenSearch servers that omit node metadata

Open
#2,145 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug untriaged
Dominant language
Java
Stars
165
Forks
250
Avg merge
1d 18h
Merged PRs (30d)
26

Description

What is the bug?

OpenSearchClient.searchShards(...) rejects an HTTP 200 response when a node omits attributes and transport_address. Applications that only need shard routing information cannot access the typed response, even when the index and shard data are present. This shape was reported from an Amazon OpenSearch Service 1.3.x domain:

{"nodes":{"node-id":{"name":"node-name","ephemeral_id":"ephemeral-id"}},"indices":{},"shards":[]}

The example is redacted and minimized; the reported response also contained index and shard data.

Reproduction

Reproduced with the published 3.9.0 and 3.10.0 clients using Jackson and JSON-B, and through the public method with a local HTTP 200 fixture server. The required fields are also unchanged on current main (1abf420c).

String json = "{\"nodes\":{\"node-id\":{\"name\":\"node-name\","
    + "\"ephemeral_id\":\"ephemeral-id\"}},\"indices\":{},\"shards\":[]}";
var mapper = new org.opensearch.client.json.jackson.JacksonJsonpMapper();
try (var parser = mapper.jsonProvider().createParser(new java.io.StringReader(json))) {
    org.opensearch.client.opensearch.core.SearchShardsResponse._DESERIALIZER
        .deserialize(parser, mapper);
}

This throws Missing required property 'NodeAttributes.attributes'. Adding "attributes":{} then throws Missing required property 'NodeAttributes.transportAddress', so both omissions need handling.

Expected behavior and proposed fix

Could the response model tolerate these two omitted metadata fields so applications can still consume the shard routing information? The proposed behavior is an empty map from node.attributes() and null from node.transportAddress(), preserving metadata when supplied. name and ephemeral_id would remain required.

Stock server serializers in 1.3.0, 1.3.20, 2.19.6, 3.0.0, and 3.8.0 emit both fields, and the current client follows that contract. This request is for client compatibility with the reported managed response. Newer managed domains have not been tested here. A related omission in task responses was handled by making node fields optional in #463, fixing #356.

The local candidate removes both requirements from the canonical and bundled schemas and regenerates the Java model. All 14 regression cases pass across Jackson and JSON-B, including populated shards, complete metadata, and round trips. The schema is also shared with _shard_stores, which is covered by the tests. I can submit linked specification and client PRs if this approach is suitable; guidance on endpoint-specific or distribution-specific treatment would also be welcome.

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at OpenSearchClient.searchShards and SearchShardsResponse._DESERIALIZER, then inspect the canonical and bundled schemas that define NodeAttributes. Regenerate the Java model after making the two metadata fields optional, and run the regression cases across Jackson and JSON-B, including populated shards, complete metadata, round trips, and the shared _shard_stores schema. Done means omitted fields deserialize as the proposed empty map and null while supplied metadata remains preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.