opensearch-project / opensearch-project/opensearch-java

[FEATURE] add support for ukrainian-analyzer and other types

Open
#1,505 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Java
Stars
165
Forks
250
Avg merge
1d 18h
Merged PRs (30d)
26

Description

Is your feature request related to a problem?

At the moment org.opensearch.client.opensearch._types.analysis.Analyzer object cannot be constructed from the JSON. Here is a modified org.opensearch.client.opensearch.types.analysis.AnalyzerDeserializerTest which can be used to reproduce the issue:

public class AnalyzerDeserializerTest extends ModelTestCase {
    @Test
    public void testDeserialize() {
        String json = """
            "type": "ukrainian_analyzer",
            "tokenizer": "ukrainian_tokenizer",
            "stemmer": "ukrainian_stemmer",
            "filter": [
                "lowercase",
                "ukrainian_stop"
            ]
        }
        """;

        Analyzer analyzer = fromJson(json, Analyzer._DESERIALIZER);
    }
}

Error that is received:

Using a Jackson mapper (rand = 21).

Unknown field 'ukrainian_analyzer' for type 'org.opensearch.client.opensearch._types.analysis.Analyzer$Builder'
jakarta.json.stream.JsonParsingException: Unknown field 'ukrainian_analyzer' for type 'org.opensearch.client.opensearch._types.analysis.Analyzer$Builder'
	at org.opensearch.client.json.ObjectDeserializer.parseUnknownField(ObjectDeserializer.java:222)
	at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:199)
	at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146)
	at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87)
	at org.opensearch.client.json.BuildFunctionDeserializer.deserialize(BuildFunctionDeserializer.java:59)
	at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55)
	at org.opensearch.client.opensearch.model.ModelTestCase.fromJson(ModelTestCase.java:117)
	at org.opensearch.client.opensearch._types.analysis.AnalyzerDeserializerTest.testDeserialize(AnalyzerDeserializerTest.java:35)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:112)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:40)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:54)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:53)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92)
	at jdk.proxy1/jdk.proxy1.$Proxy4.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:183)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:132)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:103)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:63)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:121)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)


In the end this breaks next opportunity to create index programmatically via Java API (pseudo code):

Analyzer analyzer = getAnalyzer();
OpenSearchClient client = getClient();

IndexSettingsAnalysis analysis = IndexSettingsAnalysis.builder().analyzer("ukrainian_analyzer", analyzer).build();
IndexSettings indexSettings = IndexSettings.builder().analysis(analysis).build();
PutIndicesSettingsRequest request = new PutIndicesSettingsRequest.Builder().settings(indexSettings).build();

client.indices().putSettings(request);
What solution would you like?

Ideally I expect that there will be a support for additional analyzers which as I see treated as non native.
Maybe some additional Analyzer.Kind value. I am not sure how to properly solve it, I am not very familiar with your code base.

What alternatives have you considered?

At the moment there is a workaround, we can manually generate HTTP request and insert there whatever we want. But first class support would be great!

OpenSearchClient client = getClient();
String json = """
            {
                "mappings": {
                    "properties": {
                        ...
                    }
                },
                "settings": {
                    "analysis": {
                        "analyzer": {
                            "ukrainian_analyzer": {
                                "type": "ukrainian",
                                "tokenizer": "ukrainian_tokenizer",
                                "stemmer": "ukrainian_stemmer",
                                "filter": [
                                    "lowercase",
                                    "ukrainian_stop"
                                ]
                            }
                        }
                    }
                }
            }
""";

RestClientTransport transport = ((RestClientTransport) client._transport());
Request request = new Request("PUT", "users-index");
request.setJsonEntity(json);

transport.restClient().performRequest(request);
Do you have any additional context?

I want to emphasize that it is stored in OpenSearch repository: https://github.com/opensearch-project/OpenSearch/tree/main/plugins/analysis-ukrainian

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated Analyzer.java and the reproduced test in AnalyzerDeserializerTest.java, then compare the supported Analyzer.Kind values with the analysis-ukrainian plugin context linked in the issue. Run the deserialization test with the ukrainian_analyzer JSON and determine how additional analyzers should be represented. Done means the analyzer deserializes and can be supplied through IndexSettingsAnalysis without the unknown-field error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.