opensearch-project / opensearch-project/data-prepper
data is not stored in the format of [<metric_name>: <value>] in OpenSearch.
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Versions:
opensearch & opensearch dashboard 2.11.1
data-prepper 2.6.1
opentelemetry-collector-contrib 0.92.0
When utilizing DataPrepper's metric pipeline to transmit data collected through OpenTelemetry to OpenSearch, the data is not stored in the format of <metric_name> : <value> in OpenSearch.
OTLE collector to collect kubelet data.
If you check in the opensearch dashboard, the metric name is stored in the name field.
{
name: <metric_name>
value: <value>
}
It seems to save as
I want to easily use the visualization features you provide.
{
<metric_name>: <value>
}
, is it possible to change it to something like this?
[name field]
[data-prepper config]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the metric pipeline with the listed Data Prepper, OpenTelemetry Collector, and OpenSearch versions, using the attached Data Prepper configuration. Inspect the resulting OpenSearch document and determine whether the metric pipeline or stored document mapping controls the name/value shape. Done means the requested metric-name-to-value representation is stored and can be used by OpenSearch visualizations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100