AbsaOSS / AbsaOSS/hyperdrive-trigger
Support for Spark 3
Đang mở
- Ngôn ngữ chính
- TypeScript
- Star
- 8
- Fork
- 5
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
Add support for Spark 3
Needed because of #717. Spark 3 jobs cannot be submitted with spark-submit 2 and vice versa.
- Support for Spark 2 still needed
- Two artefacts should be created and deployed, one for spark 2, one for spark 3
- If all used APIs are compatible, then just add a profile so hyperdrive-trigger can be built against spark 2, and spark 3 resp.
- Otherwise, two branches can be maintained
-
# Preparation work
Use minimal app spark launcher (not full blown hyperdrive-trigger) to test spark 3 compatibility (with EMR)
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.