Upgrade to EMR 7.13.0
- Dominant language
- Java
- Stars
- 107
- Forks
- 29
- Avg merge
- 19h 46m
- Merged PRs (30d)
- 141
Description
See https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-app-versions-7.x.html
We'll need to match the versions of all the libraries listed there.
We'll need to adjust the version of the AWS SDK used for bulk import (`aws-java-sdk-v2.bulk-import.version` in the root pom.xml), to match the version in EMR. See the comment on that property in the pom.xml for how to find the right version.
We'll need to match the versions set for EMR in the instance property default values.
We'll need to update the base Spark Docker image used for bulk import on EKS.
In our root pom.xml, we manage the versions of a lot of the transitive dependencies of Spark. When we upgrade Spark to the new version, we'll need to go through every one of them to make sure we're not keeping back the version of a library for the sake of the old version of Spark.
Contributor guide
Assessment
This issue has not been assessed yet.