opensearch-project / opensearch-project/opensearch-java
[FEATURE] add support for ukrainian-analyzer and other types
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 165
- Forks
- 250
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 26
Description
Is your feature request related to a problem?
At the moment org.opensearch.client.opensearch._types.analysis.Analyzer object cannot be constructed from the JSON. Here is a modified org.opensearch.client.opensearch.types.analysis.AnalyzerDeserializerTest which can be used to reproduce the issue:
public class AnalyzerDeserializerTest extends ModelTestCase {
@Test
public void testDeserialize() {
String json = """
"type": "ukrainian_analyzer",
"tokenizer": "ukrainian_tokenizer",
"stemmer": "ukrainian_stemmer",
"filter": [
"lowercase",
"ukrainian_stop"
]
}
""";
Analyzer analyzer = fromJson(json, Analyzer._DESERIALIZER);
}
}
Error that is received:
Using a Jackson mapper (rand = 21).
Unknown field 'ukrainian_analyzer' for type 'org.opensearch.client.opensearch._types.analysis.Analyzer$Builder'
jakarta.json.stream.JsonParsingException: Unknown field 'ukrainian_analyzer' for type 'org.opensearch.client.opensearch._types.analysis.Analyzer$Builder'
at org.opensearch.client.json.ObjectDeserializer.parseUnknownField(ObjectDeserializer.java:222)
at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:199)
at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146)
at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87)
at org.opensearch.client.json.BuildFunctionDeserializer.deserialize(BuildFunctionDeserializer.java:59)
at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55)
at org.opensearch.client.opensearch.model.ModelTestCase.fromJson(ModelTestCase.java:117)
at org.opensearch.client.opensearch._types.analysis.AnalyzerDeserializerTest.testDeserialize(AnalyzerDeserializerTest.java:35)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:112)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:40)
at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:54)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:53)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92)
at jdk.proxy1/jdk.proxy1.$Proxy4.processTestClass(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:183)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:132)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:103)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:63)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:121)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
In the end this breaks next opportunity to create index programmatically via Java API (pseudo code):
Analyzer analyzer = getAnalyzer();
OpenSearchClient client = getClient();
IndexSettingsAnalysis analysis = IndexSettingsAnalysis.builder().analyzer("ukrainian_analyzer", analyzer).build();
IndexSettings indexSettings = IndexSettings.builder().analysis(analysis).build();
PutIndicesSettingsRequest request = new PutIndicesSettingsRequest.Builder().settings(indexSettings).build();
client.indices().putSettings(request);
What solution would you like?
Ideally I expect that there will be a support for additional analyzers which as I see treated as non native.
Maybe some additional Analyzer.Kind value. I am not sure how to properly solve it, I am not very familiar with your code base.
What alternatives have you considered?
At the moment there is a workaround, we can manually generate HTTP request and insert there whatever we want. But first class support would be great!
OpenSearchClient client = getClient();
String json = """
{
"mappings": {
"properties": {
...
}
},
"settings": {
"analysis": {
"analyzer": {
"ukrainian_analyzer": {
"type": "ukrainian",
"tokenizer": "ukrainian_tokenizer",
"stemmer": "ukrainian_stemmer",
"filter": [
"lowercase",
"ukrainian_stop"
]
}
}
}
}
}
""";
RestClientTransport transport = ((RestClientTransport) client._transport());
Request request = new Request("PUT", "users-index");
request.setJsonEntity(json);
transport.restClient().performRequest(request);
Do you have any additional context?
I want to emphasize that it is stored in OpenSearch repository: https://github.com/opensearch-project/OpenSearch/tree/main/plugins/analysis-ukrainian
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the generated Analyzer.java and the reproduced test in AnalyzerDeserializerTest.java, then compare the supported Analyzer.Kind values with the analysis-ukrainian plugin context linked in the issue. Run the deserialization test with the ukrainian_analyzer JSON and determine how additional analyzers should be represented. Done means the analyzer deserializes and can be supplied through IndexSettingsAnalysis without the unknown-field error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100