opensearch-project / opensearch-project/data-prepper

Support alternate CloudWatch configurations for monitoring Data Prepper

Open
#4,465 0 comments 0 reactions 1 assignee View on GitHub

@dlvenable is already working on this.

Since Apr 30, 2024.

enhancement
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Is your feature request related to a problem? Please describe.

Data Prepper supports CloudWatch as a metric destination for its own metrics. The data-prepper-core project includes a cloudwatch.properties file.

It is good that Data Prepper is pre-configured for CloudWatch so that no additional configuration is needed.

However, the current approach provides some difficulties:

  • It is difficult to override these configurations
  • When creating a custom Data Prepper distribution, the file in this jar may conflict with other similar files that teams provide.

Describe the solution you'd like

Move the cloudwatch.properties file out of data-prepper-core.jar. Instead, include it in the config directory of Data Prepper at config/cloudwatch.properties. With this, users can change this file and get their own custom configurations.

Describe alternatives you've considered (Optional)

There may still be some value in including the cloudwatch.properties file in the jars. If so, we may wish to move it into data-prepper-main.jar since that jar is more opinionated on how Data Prepper is run.

Additional context

See #4464 for another file that is in data-prepper-core.jar and causing difficulty for custom distributions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.