opensearch-project / opensearch-project/data-prepper
Metric for CloudWatch Logs LogEvent size summary
@kkondaka is already working on this.
Since Oct 14, 2025.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
The cloudwatch_logs sink writes each Data Prepper event to a CloudWatch Logs LogEvent. If the size exceeds the maximum allowed size by CloudWatch Logs, the sink drops that data.
There is currently no visibility into these drops beyond the DLQ.
Describe the solution you'd like
I'd like two metrics:
- A summary of CloudWatch Logs LogEvent sizes.
- A counter for LogEvents that were dropped before the API call for any reason.
Describe alternatives you've considered (Optional)
We could have a specific metric for events dropped due to size, but this could result in many metrics that only count. Having the size will provide more context to users when failure scenarios occur.
Additional context
N/A
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.