opensearch-project / opensearch-project/opensearch-java

[BUG] custom analyzers not supported

Open
#1,510 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
165
Forks
250
Avg merge
1d 18h
Merged PRs (30d)
26

Description

What is the bug?

while there's the generated Analyzer class it does not list all analyzers (e.g. phone and phone-search, introduced with 2.18, are not yet present there) and i'm also not sure how that should be used with spring-data-opensearch' @Field annotation (which takes a string, not an enum).
when using a string in the annotation, e.g. @Field(type = FieldType.Text, analyzer = "phone", searchAnalyzer = "phone-search") this works fine with ORHLC, but it fails when switching to opensearch-java:

Missing required property 'Builder.<variant kind>'
org.opensearch.client.util.MissingRequiredPropertyException: Missing required property 'Builder.<variant kind>'
	at app//org.opensearch.client.util.ApiTypeHelper.requireNonNull(ApiTypeHelper.java:90)
	at app//org.opensearch.client.opensearch._types.analysis.Analyzer.<init>(Analyzer.java:131)
	at app//org.opensearch.client.opensearch._types.analysis.Analyzer.<init>(Analyzer.java:53)
	at app//org.opensearch.client.opensearch._types.analysis.Analyzer$Builder.build(Analyzer.java:585)
	at app//org.opensearch.client.json.BuildFunctionDeserializer.deserialize(BuildFunctionDeserializer.java:60)
	at app//org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55)
	at app//org.opensearch.client.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:369)
	at app//org.opensearch.client.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:355)
	at app//org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87)
	at app//org.opensearch.client.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:81)
	at app//org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:185)
	at app//org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146)
	at app//org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87)
	at app//org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:91)
	at app//org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55)
	at app//org.opensearch.client.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:81)
	at app//org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:185)
	at app//org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146)
	at app//org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87)
	at app//org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:91)
	at app//org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55)
	at app//org.opensearch.data.client.osc.JsonpUtils.fromJson(JsonpUtils.java:67)
	at app//org.opensearch.data.client.osc.TypeUtils.indexSettings(TypeUtils.java:476)
	at app//org.opensearch.data.client.osc.RequestConverter.indicesCreateRequest(RequestConverter.java:255)
	at app//org.opensearch.data.client.osc.IndicesTemplate.doCreate(IndicesTemplate.java:138)
	at app//org.opensearch.data.client.osc.IndicesTemplate.create(IndicesTemplate.java:115)
	at app//com.avaloq.rmw.globalsearch.configuration.AutoIndexMappingConfiguration.createIndexMappingsOnStartup(AutoIndexMappingConfiguration.java:94)
	at app//com.avaloq.rmw.globalsearch.configuration.AutoIndexMappingConfiguration.runOnStartup(AutoIndexMappingConfiguration.java:59)
	at java.base@21.0.5/java.lang.reflect.Method.invoke(Method.java:580)
	at app//org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
	at app//org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at app//org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at app//org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
	at app//com.avaloq.rmw.globalsearch.aspect.LoggingAspect.logMethodStartAndEndExecution(LoggingAspect.java:27)
	at java.base@21.0.5/java.lang.reflect.Method.invoke(Method.java:580)
	at app//org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:642)
	at app//org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:632)
	at app//org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:71)
	at app//org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at app//org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at app//org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at app//org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:727)
	at app//com.avaloq.rmw.globalsearch.configuration.AutoIndexMappingConfiguration$$SpringCGLIB$$0.runOnStartup(<generated>)
	at java.base@21.0.5/java.lang.reflect.Method.invoke(Method.java:580)
	at app//org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:382)
	at app//org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:254)
	at app//org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:173)
	at app//org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:185)
	at app//org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:178)
	at app//org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:156)
	at app//org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:454)
	at app//org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:387)
	at app//org.springframework.boot.context.event.EventPublishingRunListener.started(EventPublishingRunListener.java:103)
	at app//org.springframework.boot.SpringApplicationRunListeners.lambda$started$5(SpringApplicationRunListeners.java:76)
	at java.base@21.0.5/java.lang.Iterable.forEach(Iterable.java:75)
	at app//org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
	at app//org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
	at app//org.springframework.boot.SpringApplicationRunListeners.started(SpringApplicationRunListeners.java:76)
	at app//org.springframework.boot.SpringApplication.run(SpringApplication.java:324)
	at app//org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137)
	at app//org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58)
	at app//org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46)
	at app//org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1461)
	at app//org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:553)
	at app//org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137)
	at app//org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:108)
	at app//org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:225)
	at app//org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:152)
	at app//org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:130)
	at app//org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:200)
	at app//org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:139)
	at app//org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:260)
	at app//org.springframework.test.context.junit.jupiter.SpringExtension.postProcessTestInstance(SpringExtension.java:160)
	at java.base@21.0.5/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base@21.0.5/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base@21.0.5/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base@21.0.5/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base@21.0.5/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
	at java.base@21.0.5/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base@21.0.5/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base@21.0.5/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.base@21.0.5/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.base@21.0.5/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base@21.0.5/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base@21.0.5/java.util.Optional.orElseGet(Optional.java:364)
	at java.base@21.0.5/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base@21.0.5/java.util.ArrayList.forEach(ArrayList.java:1596)

the error message is completely unclear and it's not obvious that this is due to the analyzer.

How can one reproduce the bug?

try to use the phone analyzer on a field using spring-data-opensearch with opensearch-java.

What is the expected behavior?

it is possible to use all analyzers, regardless of whether they are built-in or custom (e.g. by defining one in settings.analysis.analyzer and using that e.g. using @Setting in spring-data-opensearch).

What is your host/environment?
  • spring-data-opensearch:1.6.2
  • opensearch-java:2.22.0
Do you have any screenshots?

n/a

Do you have any additional context?

#779 is related to this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with java-client/src/generated/java/org/opensearch/client/opensearch/_types/analysis/Analyzer.java and trace deserialization through TypeUtils.indexSettings and RequestConverter.indicesCreateRequest. Reproduce the phone analyzer case with spring-data-opensearch, then verify that built-in and custom analyzer names can be used without the unclear MissingRequiredPropertyException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.