Azure / Azure/azure-quickstart-templates
Use HDInsight Spark on Linux cluster instead of custom script action to install Spark manually
- Dominant language
- Bicep
- Stars
- 14.9k
- Forks
- 16.2k
- Avg merge
- 6d 21h
- Merged PRs (30d)
- 6
Description
[hdInsight-apache-spark](https://github.com/Azure/azure-quickstart-templates/tree/master/hdInsight-apache-spark)
Since Spark is now GA on HDInsight Linux clusters, it may be more helpful to create a Spark cluster in the template instead of creating a vanilla HDInsight cluster and using script action to manually install the Spark binaries.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the hdInsight-apache-spark template linked in the issue and inspect how it creates the cluster and installs Spark. Replace the manual Spark installation approach with the native HDInsight Linux Spark cluster configuration, then verify that the template deploys a Spark cluster without the custom script action.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, spark
- Domain
- cloud, data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100