opensearch-project / opensearch-project/data-prepper

Remove the AWS SDK v1 (again)

Open
#4,017 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

maintenance
Dominant language
Java
Stars
374
Forks
355
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Is your feature request related to a problem? Please describe.

Data Prepper should use the AWS SDK v2 exclusively. This will reduce the number of dependencies it pulls in and allow all projects to use the AWS extension.

We did this once before: #1562

Describe the solution you'd like

First, remove all existing dependencies:

https://github.com/opensearch-project/data-prepper/blob/d501af69c10b93efbb0c68abeeda4f5023323e18/data-prepper-plugins/translate-processor/build.gradle#L14-L15

https://github.com/opensearch-project/data-prepper/blob/334239bb662f6176c2ab6c3423b8f30214355488/data-prepper-plugins/kafka-plugins/build.gradle#L49

https://github.com/opensearch-project/data-prepper/blob/a9e419af24810cad29a950205376bc5fc2b89391/data-prepper-plugins/http-sink/build.gradle#L29

Second, I'd like a rule in Gradle that disallows using the AWS SDK v1.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the three referenced build.gradle files in translate-processor, kafka-plugins, and http-sink, and inspect the AWS SDK dependencies there. Remove the AWS SDK v1 dependencies and add a Gradle rule that rejects AWS SDK v1 usage. Done means the listed dependencies are gone and the build prevents new v1 dependencies.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
build-system
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.