opensearch-project / opensearch-project/data-prepper
Enhance SQS sink logging for better troubleshooting
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
When debugging SQS sink issues, there is insufficient logging to understand why messages are being dropped, sent to DLQ, or rejected.
Describe the solution you'd like
Add comprehensive debug and info level logging to the SQS sink, specifically in SqsSinkBatch.java and related classes:
-
Size limit violations : Log when messages are rejected due to exceeding configured size limits
-
DLQ routing : Log the specific reason why messages are sent to DLQ with error details
-
SQS API errors : Log detailed error information for failed SQS requests
Describe alternatives you've considered (Optional)
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading SqsSinkBatch.java and the related SQS sink classes to trace message rejection, DLQ routing, and failed SQS requests. Identify the existing handling for each path, then add debug or info logs with the requested reasons and error details. Done means these three outcomes produce useful troubleshooting logs without changing routing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100