opensearch-project / opensearch-project/data-prepper

[BUG] [S3 source] Pause in SQS processing when there is an issue in reading S3 object

Open
#4,569 2 comments 0 reactions 1 assignee View on GitHub

@sb2k16 is already working on this.

Since Nov 5, 2024.

bug Priority-High
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug
Data-prepper S3 source pauses SQS processing with exponential backoff when there is an issue in reading S3 object such as corrupted parquet file.

To Reproduce
Steps to reproduce the behavior:

  1. Creat a data-prepper pipeline with S3 source
  2. upload corrupted s3 objects to the bucket
  3. Observe data-prepper logs with message "Pausing SQS processing for XXX seconds due to an error in processing."

Expected behavior
S3 source plugin should skip corrupted objects and process next object without delay. S3 source should backoff only when there is an error with the SQS processing itself.

Additional context
Add any other context about the problem here.

[s3-source-sqs-2] ERROR org.opensearch.dataprepper.plugins.source.s3.SqsWorker - Error processing from S3: java.io.IOException: can not read class org.apache.parquet.format.FileMetaData: Required field 'num_rows' was not found in serialized data! Struct: org.apache.parquet.format.FileMetaData$FileMetaDataStandardScheme@62202421. Retrying with exponential backoff.
[s3-source-sqs-2] INFO  org.opensearch.dataprepper.plugins.source.s3.SqsWorker - Pausing SQS processing for 19.858 seconds due to an error in processing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.