opensearch-project / opensearch-project/data-prepper
[BUG] Exception when OTel metrics data is sent in JSON format with `instrumentationLibrary` field
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Description:
As part of the changes introduced in #5434, the OTEL protobuf specification was updated. While the system remains backward compatible when using Protobuf, issues arise when the data is sent in JSON format.
When sample JSON data includes instrumentationLibraryMetrics with an empty instrumentationLibrary and a list of metrics like this:
"instrumentationLibraryMetrics": [
{
"instrumentationLibrary": {},
"metrics": [
{
"name": "counter-int"
}
]
}
]
The following exception is thrown:
org.opensearch.dataprepper.GrpcRequestExceptionHandler - Unexpected exception handling gRPC request
com.google.protobuf.InvalidProtocolBufferException: Expected start of object, got: [
at org.curioswitch.common.protobuf.json.TypeSpecificMarshaller.mergeValue(TypeSpecificMarshaller.java:66) ~[protobuf-jackson-2.5.0.jar:?]
Expected Behavior:
Data Prepper should handle empty instrumentationLibrary objects gracefully in JSON mode, without throwing an exception. At a minimum, a more user-friendly error message would help in identifying and correcting malformed input.
Steps to Reproduce:
- Send OTEL metrics in JSON format with an empty
instrumentationLibraryobject. - Observe the thrown
InvalidProtocolBufferException.
Sample pipeline configuration:
test-pipeline:
source:
otel_metrics_source:
ssl: false
port: 4317
path: /log/ingest
unframed_requests: true
sink:
- stdout:
curl -X POST http://localhost:4317/log/ingest \
-H "Content-Type: application/json" \
-u admin:securepass \
-d '{
"resourceMetrics": [
{
"instrumentationLibraryMetrics": [
{
"instrumentationLibrary": {},
"metrics": [
{
"name": "cpu_usage",
"sum": {
"dataPoints": [
{
"asDouble": 75.5
}
]
}
}
]
}
]
}
]
}'
Environment:
- Data Prepper version: [version that includes PR #5434]
- Data format: JSON
- Libraries:
protobuf-jackson-2.5.0
Suggested Fix:
Investigate JSON deserialization logic for instrumentationLibrary and handle cases where the field is present but empty.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the provided pipeline and curl request, then trace it from the gRPC request handler into the JSON deserialization logic for instrumentationLibrary. Confirm how an empty field is handled and verify that the request no longer throws an unhelpful exception or produces a clearer error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100