opensearch-project / opensearch-project/data-prepper
Support alternate CloudWatch configurations for monitoring Data Prepper
@dlvenable is already working on this.
Since Apr 30, 2024.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
Data Prepper supports CloudWatch as a metric destination for its own metrics. The data-prepper-core project includes a cloudwatch.properties file.
It is good that Data Prepper is pre-configured for CloudWatch so that no additional configuration is needed.
However, the current approach provides some difficulties:
- It is difficult to override these configurations
- When creating a custom Data Prepper distribution, the file in this jar may conflict with other similar files that teams provide.
Describe the solution you'd like
Move the cloudwatch.properties file out of data-prepper-core.jar. Instead, include it in the config directory of Data Prepper at config/cloudwatch.properties. With this, users can change this file and get their own custom configurations.
Describe alternatives you've considered (Optional)
There may still be some value in including the cloudwatch.properties file in the jars. If so, we may wish to move it into data-prepper-main.jar since that jar is more opinionated on how Data Prepper is run.
Additional context
See #4464 for another file that is in data-prepper-core.jar and causing difficulty for custom distributions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.