Azure / Azure/azure-quickstart-templates

Use HDInsight Spark on Linux cluster instead of custom script action to install Spark manually

Open
#2,257 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Bicep
Stars
14.9k
Forks
16.2k
Avg merge
6d 21h
Merged PRs (30d)
6

Description

[hdInsight-apache-spark](https://github.com/Azure/azure-quickstart-templates/tree/master/hdInsight-apache-spark)

Since Spark is now GA on HDInsight Linux clusters, it may be more helpful to create a Spark cluster in the template instead of creating a vanilla HDInsight cluster and using script action to manually install the Spark binaries.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the hdInsight-apache-spark template linked in the issue and inspect how it creates the cluster and installs Spark. Replace the manual Spark installation approach with the native HDInsight Linux Spark cluster configuration, then verify that the template deploys a Spark cluster without the custom script action.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, spark
Domain
cloud, data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.