opensearch-project / opensearch-project/data-prepper
[BUG] OTEL - Activity Tag: Array is not supported
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Describe the bug
In my application, if I set a tag with a array of int as value, the data-prepper is unable to process the ressourcespan.
To Reproduce
Steps to reproduce the behavior:
In a c# .net core console, add OpenTelemetry package and opentelemetry.exporter.opentelemetryprotocol
Add :
var MyActivitySource = new ActivitySource(serviceName);
using var activity = MyActivitySource.StartActivity("SayHello");
activity?.SetTag("baz", new int[] { 1, 2, 3 });
Setup the OtelCollector to send data to the Data-prepper:
Config:
receivers:
otlp:
protocols:
grpc:
#endpoint: 0.0.0.0:55680
exporters:
otlp/2:
endpoint: localhost:21890
tls:
insecure: true
insecure_skip_verify: true
logging:
service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging, otlp/2]
when the trace reach the server, it can't be consumed as the "bar" attributes is set multiples times (Array data).
Expected behavior
No errors.
Error Logs
2022-04-11T15:20:06,952 [raw-pipeline-processor-worker-5-thread-1] ERROR com.amazon.dataprepper.plugins.prepper.oteltrace.OTelTraceRawPrepper - Unable to process invalid ResourceSpan resource {
data-prepper | attributes {
data-prepper | key: "service.name"
data-prepper | value {
data-prepper | string_value: "MDS.TestSerilog.Test"
data-prepper | }
data-prepper | }
data-prepper | attributes {
data-prepper | key: "service.version"
data-prepper | value {
data-prepper | string_value: "1.0.0"
data-prepper | }
data-prepper | }
data-prepper | attributes {
data-prepper | key: "service.instance.id"
data-prepper | value {
data-prepper | string_value: "be537731-e6c8-48b8-8202-2d50301b61c9"
data-prepper | }
data-prepper | }
data-prepper | }
data-prepper | instrumentation_library_spans {
data-prepper | instrumentation_library {
data-prepper | name: "MDS.TestSerilog.Test"
data-prepper | }
data-prepper | spans {
data-prepper | trace_id: "\355\273\256\213\nc9D\253\211\324\240lD\220s"
data-prepper | span_id: "\307) R\276\247\244A"
data-prepper | name: "SayHello"
data-prepper | kind: SPAN_KIND_INTERNAL
data-prepper | start_time_unix_nano: 1649690403941588600
data-prepper | end_time_unix_nano: 1649690403942298100
data-prepper | attributes {
data-prepper | key: "foo"
data-prepper | value {
data-prepper | int_value: 1
data-prepper | }
data-prepper | }
data-prepper | attributes {
data-prepper | key: "bar"
data-prepper | value {
data-prepper | string_value: "Hello, World32!"
data-prepper | }
data-prepper | }
data-prepper | attributes {
data-prepper | key: "baz"
data-prepper | value {
data-prepper | int_value: 1
data-prepper | }
data-prepper | }
data-prepper | attributes {
data-prepper | key: "baz"
data-prepper | value {
data-prepper | int_value: 2
data-prepper | }
data-prepper | }
data-prepper | attributes {
data-prepper | key: "baz"
data-prepper | value {
data-prepper | int_value: 3
data-prepper | }
data-prepper | }
data-prepper | status {
data-prepper | }
data-prepper | }
data-prepper | }
data-prepper | :
data-prepper | java.lang.IllegalStateException: Duplicate key span.attributes.baz (attempted merging values 1 and 2)
Environment (please complete the following information):
- OS: Windows / linux - Data-preper is running on latest docker image
- Version: latest as of 11 April 2022
Thanks for the help!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the OTelTraceRawPrepper entry point shown in the error log and reproduce the issue using the provided C# Activity example and collector configuration. Confirm that a span containing repeated baz attributes from an integer array is processed without the duplicate-key failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, java
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100