opensearch-project / opensearch-project/data-prepper
Add new index-type ecs to support ecs formatted index name for metrics, logs, etc.
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
Add new index-type ecs to support ecs formatted index name for metrics, logs, etc. based on the naming scheme mentioned here -
https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme
Describe the solution you'd like
I think the following solution is simple and easy
Add new index type to the existing index-types in opensearch sink
index-type: ecs
# index-name: <type>-<dataset>-<namespace>
index-name: metrics-ngnix.access-xyz
Describe alternatives you've considered (Optional)
Alternative approach is
index-type: ecs
type-name: metrics
dataset-name: ngnix.access
namespace-name: xyz
Additional context
Need to create an index mapping for different types. To start with, index mapping for histogram metric type should be created as per OTEL format
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the OpenSearch sink's existing index-type handling and review the linked Elastic data-stream naming scheme. Define how the ecs type should produce the requested metrics, logs, and related names, then verify the histogram metric mapping follows the stated OTEL format.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100