opensearch-project / opensearch-project/data-prepper

[BUG] OTEL - Activity Tag: Array is not supported

Open
#1,307 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backlog bug enhancement
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug
In my application, if I set a tag with a array of int as value, the data-prepper is unable to process the ressourcespan.

To Reproduce
Steps to reproduce the behavior:
In a c# .net core console, add OpenTelemetry package and opentelemetry.exporter.opentelemetryprotocol

Add :

                var MyActivitySource = new ActivitySource(serviceName);

                using var activity = MyActivitySource.StartActivity("SayHello");
                activity?.SetTag("baz", new int[] { 1, 2, 3 });

Setup the OtelCollector to send data to the Data-prepper:
Config:

receivers:
  otlp:
    protocols:
      grpc:
        #endpoint: 0.0.0.0:55680

exporters:
  otlp/2:
    endpoint: localhost:21890
    tls:
      insecure: true
      insecure_skip_verify: true
  logging:

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [logging, otlp/2]

when the trace reach the server, it can't be consumed as the "bar" attributes is set multiples times (Array data).

Expected behavior
No errors.

Error Logs

2022-04-11T15:20:06,952 [raw-pipeline-processor-worker-5-thread-1] ERROR com.amazon.dataprepper.plugins.prepper.oteltrace.OTelTraceRawPrepper - Unable to process invalid ResourceSpan resource {
data-prepper              |   attributes {
data-prepper              |     key: "service.name"
data-prepper              |     value {
data-prepper              |       string_value: "MDS.TestSerilog.Test"
data-prepper              |     }
data-prepper              |   }
data-prepper              |   attributes {
data-prepper              |     key: "service.version"
data-prepper              |     value {
data-prepper              |       string_value: "1.0.0"
data-prepper              |     }
data-prepper              |   }
data-prepper              |   attributes {
data-prepper              |     key: "service.instance.id"
data-prepper              |     value {
data-prepper              |       string_value: "be537731-e6c8-48b8-8202-2d50301b61c9"
data-prepper              |     }
data-prepper              |   }
data-prepper              | }
data-prepper              | instrumentation_library_spans {
data-prepper              |   instrumentation_library {
data-prepper              |     name: "MDS.TestSerilog.Test"
data-prepper              |   }
data-prepper              |   spans {
data-prepper              |     trace_id: "\355\273\256\213\nc9D\253\211\324\240lD\220s"
data-prepper              |     span_id: "\307) R\276\247\244A"
data-prepper              |     name: "SayHello"
data-prepper              |     kind: SPAN_KIND_INTERNAL
data-prepper              |     start_time_unix_nano: 1649690403941588600
data-prepper              |     end_time_unix_nano: 1649690403942298100
data-prepper              |     attributes {
data-prepper              |       key: "foo"
data-prepper              |       value {
data-prepper              |         int_value: 1
data-prepper              |       }
data-prepper              |     }
data-prepper              |     attributes {
data-prepper              |       key: "bar"
data-prepper              |       value {
data-prepper              |         string_value: "Hello, World32!"
data-prepper              |       }
data-prepper              |     }
data-prepper              |     attributes {
data-prepper              |       key: "baz"
data-prepper              |       value {
data-prepper              |         int_value: 1
data-prepper              |       }
data-prepper              |     }
data-prepper              |     attributes {
data-prepper              |       key: "baz"
data-prepper              |       value {
data-prepper              |         int_value: 2
data-prepper              |       }
data-prepper              |     }
data-prepper              |     attributes {
data-prepper              |       key: "baz"
data-prepper              |       value {
data-prepper              |         int_value: 3
data-prepper              |       }
data-prepper              |     }
data-prepper              |     status {
data-prepper              |     }
data-prepper              |   }
data-prepper              | }
data-prepper              |  :
data-prepper              | java.lang.IllegalStateException: Duplicate key span.attributes.baz (attempted merging values 1 and 2)

Environment (please complete the following information):

  • OS: Windows / linux - Data-preper is running on latest docker image
  • Version: latest as of 11 April 2022

Thanks for the help!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the OTelTraceRawPrepper entry point shown in the error log and reproduce the issue using the provided C# Activity example and collector configuration. Confirm that a span containing repeated baz attributes from an integer array is processed without the duplicate-key failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, java
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.