opensearch-project / opensearch-project/data-prepper

[BUG] jaeger HotROD example is throwing exceptions by default.

Open
#941 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backlog bug ease-of-use
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug
jaeger HotROD example throwing exceptions by default. traces are still visible on OpenSearch.

To Reproduce
Run the jaeger-hotrod example

Expected behavior
No exceptions on the example run

Environment (please complete the following information):

  • OS: macos, docker
  • Version: docker engine v20.10.8

Additional context
exceptions:

data-prepper      | 2022-01-26T08:36:19,132 [main] ERROR com.amazon.dataprepper.plugin.PluginCreator - Encountered exception while instantiating the plugin OpenSearchSink
data-prepper      | java.lang.reflect.InvocationTargetException: null
data-prepper      |     at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
data-prepper      |     at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
data-prepper      |     at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
data-prepper      |     at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
data-prepper      |     at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
data-prepper      |     at com.amazon.dataprepper.plugin.PluginCreator.newPluginInstance(PluginCreator.java:38) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.plugin.DefaultPluginFactory.loadPlugin(DefaultPluginFactory.java:66) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:163) ~[data-prepper.jar:1.2.1]
data-prepper      |     at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) [?:?]
data-prepper      |     at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) [?:?]
data-prepper      |     at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) [?:?]
data-prepper      |     at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) [?:?]
data-prepper      |     at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) [?:?]
data-prepper      |     at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?]
data-prepper      |     at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) [?:?]
data-prepper      |     at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:105) [data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:70) [data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:129) [data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:33) [data-prepper.jar:1.2.1]
data-prepper      | Caused by: java.lang.RuntimeException: Connection refused
data-prepper      |     at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:91) ~[data-prepper.jar:1.2.1]
data-prepper      |     ... 19 more
data-prepper      | Caused by: java.net.ConnectException: Connection refused
data-prepper      |     at org.opensearch.client.RestClient.extractAndWrapCause(RestClient.java:892) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestClient.performRequest(RestClient.java:296) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestClient.performRequest(RestClient.java:283) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1394) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.plugins.sink.opensearch.index.IndexManager.checkISMEnabled(IndexManager.java:45) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.initialize(OpenSearchSink.java:99) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:88) ~[data-prepper.jar:1.2.1]
data-prepper      |     ... 19 more
data-prepper      | Caused by: java.net.ConnectException: Connection refused
data-prepper      |     at sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]
data-prepper      |     at sun.nio.ch.Net.pollConnectNow(Net.java:660) ~[?:?]
data-prepper      |     at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:875) ~[?:?]
data-prepper      |     at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[data-prepper.jar:1.2.1]
data-prepper      |     at java.lang.Thread.run(Thread.java:832) ~[?:?]
data-prepper      | 2022-01-26T08:36:19,142 [main] ERROR com.amazon.dataprepper.parser.PipelineParser - Construction of pipeline components failed, skipping building of pipeline [service-map-pipeline] and its connected pipelines
data-prepper      | com.amazon.dataprepper.model.plugin.PluginInvocationException: Exception throw from the plugin'OpenSearchSink'.
data-prepper      |     at com.amazon.dataprepper.plugin.PluginCreator.newPluginInstance(PluginCreator.java:44) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.plugin.DefaultPluginFactory.loadPlugin(DefaultPluginFactory.java:66) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.parser.PipelineParser.buildSinkOrConnector(PipelineParser.java:163) ~[data-prepper.jar:1.2.1]
data-prepper      |     at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
data-prepper      |     at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
data-prepper      |     at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
data-prepper      |     at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
data-prepper      |     at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
data-prepper      |     at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
data-prepper      |     at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
data-prepper      |     at com.amazon.dataprepper.parser.PipelineParser.buildPipelineFromConfiguration(PipelineParser.java:105) [data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.parser.PipelineParser.parseConfiguration(PipelineParser.java:70) [data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.DataPrepper.execute(DataPrepper.java:129) [data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.DataPrepperExecute.main(DataPrepperExecute.java:33) [data-prepper.jar:1.2.1]
data-prepper      | Caused by: java.lang.reflect.InvocationTargetException
data-prepper      |     at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
data-prepper      |     at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
data-prepper      |     at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
data-prepper      |     at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
data-prepper      |     at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
data-prepper      |     at com.amazon.dataprepper.plugin.PluginCreator.newPluginInstance(PluginCreator.java:38) ~[data-prepper.jar:1.2.1]
data-prepper      |     ... 13 more
data-prepper      | Caused by: java.lang.RuntimeException: Connection refused
data-prepper      |     at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:91) ~[data-prepper.jar:1.2.1]
data-prepper      |     at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
data-prepper      |     at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
data-prepper      |     at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
data-prepper      |     at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
data-prepper      |     at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
data-prepper      |     at com.amazon.dataprepper.plugin.PluginCreator.newPluginInstance(PluginCreator.java:38) ~[data-prepper.jar:1.2.1]
data-prepper      |     ... 13 more
data-prepper      | Caused by: java.net.ConnectException: Connection refused
data-prepper      |     at org.opensearch.client.RestClient.extractAndWrapCause(RestClient.java:892) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestClient.performRequest(RestClient.java:296) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestClient.performRequest(RestClient.java:283) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1394) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1364) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1334) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.opensearch.client.ClusterClient.getSettings(ClusterClient.java:106) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.plugins.sink.opensearch.index.IndexManager.checkISMEnabled(IndexManager.java:45) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.initialize(OpenSearchSink.java:99) ~[data-prepper.jar:1.2.1]
data-prepper      |     at com.amazon.dataprepper.plugins.sink.opensearch.OpenSearchSink.<init>(OpenSearchSink.java:88) ~[data-prepper.jar:1.2.1]
data-prepper      |     at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
data-prepper      |     at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
data-prepper      |     at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
data-prepper      |     at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
data-prepper      |     at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
data-prepper      |     at com.amazon.dataprepper.plugin.PluginCreator.newPluginInstance(PluginCreator.java:38) ~[data-prepper.jar:1.2.1]
data-prepper      |     ... 13 more
data-prepper      | Caused by: java.net.ConnectException: Connection refused
data-prepper      |     at sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]
data-prepper      |     at sun.nio.ch.Net.pollConnectNow(Net.java:660) ~[?:?]
data-prepper      |     at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:875) ~[?:?]
data-prepper      |     at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221) ~[data-prepper.jar:1.2.1]
data-prepper      |     at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[data-prepper.jar:1.2.1]
data-prepper      |     at java.lang.Thread.run(Thread.java:832) ~[?:?]
data-prepper      | 2022-01-26T08:36:19,146 [main] ERROR       com.amazon.dataprepper.DataPrepper - No valid pipeline is available for execution, exiting

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the examples/jaeger-hotrod example with the documented Docker setup and capture when the OpenSearchSink connection is refused. Read OpenSearchSink and IndexManager at the stack-trace entry points, then verify the example completes without exceptions while traces remain visible in OpenSearch.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.