opensearch-project / opensearch-project/data-prepper
Run S3 source integration tests as GHA
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
The s3 source integration tests do not run automatically. Thus, PRs may be breaking behavior without our knowing.
Describe the solution you'd like
Run the s3 source integration tests as part of the GitHub Action. To support this, create an S3 bucket and SQS queue in the testing CDK.
It may also be advantageous to update the tests to write to a key prefix depending on the use-case: SQS notifications versus S3 scan. Then the SQS queue can trigger for events only a known key prefix. This way we don't need a bucket per use-case.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the s3 source integration tests, the testing CDK, and the GitHub Actions workflows. Determine how the tests use S3 notifications versus S3 scanning, then add the required S3 bucket and SQS queue and wire the tests into GHA; done means the integration tests run automatically in the action.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, github-actions, java
- Domain
- ci-cd, cloud, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100