[BUG] Prometheus Sink - Posting to AMP : FailedHttpResponseInterceptor is wrongly interpreting 200

Open
#3,291 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
50/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
aws, java, prometheus

Research direction

Start with FailedHttpResponseInterceptor.java at line 29, then trace its use from PrometheusSinkService.java at line 310. Reproduce the issue by running Dataprepper and ArmeriaExportMetrics.java against the AMP endpoint. Done means a successful HTTP 200 is accepted without an exception while the expected error handling remains intact.

Written by the indexing model from the issue text.

Description

bug

Describe the bug

HTTP Sink - Posting to AMP : FailedHttpResponseInterceptor is wrongly interpreting 200 as error and throwing exception which is causing pipeline to break

To Reproduce
Steps to reproduce the behavior:

  1. Run Dataprepper
  2. Generate metrics by running ArmeriaExportMetrics.java
  3. Data is successfully posted to AMP endpoint but still see the stack trace
  4. See error

java.io.IOException: url: https://aps-workspaces.us-east-1.amazonaws.com/workspaces/ws-a13b8ab8-b903-4b6a-8fe9-6722053c2469/api/v1/remote_write , status code: 200
at org.opensearch.dataprepper.plugins.sink.prometheus.FailedHttpResponseInterceptor.process(FailedHttpResponseInterceptor.java:29) ~[prometheus-sink-2.5.0-SNAPSHOT.jar:?]
at org.apache.hc.core5.http.protocol.DefaultHttpProcessor.process(DefaultHttpProcessor.java:117) ~[httpcore5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:119) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170) ~[httpclient5-5.2.jar:5.2]
at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:106) ~[httpclient5-5.2.jar:5.2]
at org.opensearch.dataprepper.plugins.sink.prometheus.service.PrometheusSinkService.pushToEndPoint(PrometheusSinkService.java:310) ~[prometheus-sink-2.5.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.sink.prometheus.service.PrometheusSinkService.lambda$output$0(PrometheusSinkService.java:180) ~[prometheus-sink-2.5.0-SNAPSHOT.jar:?]
at java.util.ArrayList.forEach(ArrayList.java:1511) ~[?:?]
at org.opensearch.dataprepper.plugins.sink.prometheus.service.PrometheusSinkService.output(PrometheusSinkService.java:145) ~[prometheus-sink-2.5.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.sink.prometheus.PrometheusSink.doOutput(PrometheusSink.java:113) ~[prometheus-sink-2.5.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.model.sink.AbstractSink.lambda$output$0(AbstractSink.java:64) ~[data-prepper-api-2.5.0-SNAPSHOT.jar:?]
at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:141) ~[micrometer-core-1.10.5.jar:1.10.5]
at org.opensearch.dataprepper.model.sink.AbstractSink.output(AbstractSink.java:64) ~[data-prepper-api-2.5.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.pipeline.Pipeline.lambda$publishToSinks$5(Pipeline.java:336) ~[data-prepper-core-2.5.0-SNAPSHOT.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]

Expected behavior

HTTP Client should handle the Status code properly and not throw exception.

Dominant language
Java
Stars
374
Forks
355
Avg merge
3d 18h
Merged PRs (30d)
8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from opensearch-project/data-prepper

All issues in opensearch-project/data-prepper

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.