Azure / Azure/azure-quickstart-templates

Deploying a HDInsight Cluster with an insufficient head node size will result in a silent head node size increase

Open
#4,915 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Bicep
Stars
14.9k
Forks
16.2k
Avg merge
6d 21h
Merged PRs (30d)
6

Description

[Base source code](https://github.com/Azure/azure-quickstart-templates/tree/master/101-hdinsight-spark-linux)

### Issue Details

As per [HDInsight doc](https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-hadoop-provision-linux-clusters#virtual-machine-sizes), it says "If you need more than 32 worker nodes in a cluster, you must select a head node size with at least **8 cores** and 14 GB of RAM.".

### Repro steps (*if necessary, delete otherwise*)
1. Change vm type:
Change `headnode.targetInstanceCount` from 2 to 1
https://github.com/Azure/azure-quickstart-templates/blob/master/101-hdinsight-spark-linux/azuredeploy.json#L100
2. Create resource group
`az group create --name "$resource_group_name" --location "East US"`
3. `az group deployment create --resource-group "$resource_group_name" --name "$deploy_name" --template-file azuredeploy.json --parameters @azuredeploy.parameters.json`
4. The deployment will succeed with
```
"computeProfile": {
"roles": [
{
"encryptDataDisks": false,
"hardwareProfile": {
"vmSize": "Standard_D12_v2"
},
"name": "headnode",
"osProfile": {
"linuxOperatingSystemProfile": {
"username": "sshuser"
}
},
"targetInstanceCount": 2 # not 1
},

```
The cluster will be launched with `D12 v2 (**x2**), D12 v2 (x5)`. As there is no prompt or warning message during the process, end users may get caught by surprise.

Contributor guide

No contributing guide indexed for this repository

Research direction

The affected template is 101-hdinsight-spark-linux/azuredeploy.json, especially the headnode.targetInstanceCount setting near line 100. Reproduce the deployment with that value set to 1 and compare the requested and resulting computeProfile values. Confirm with maintainers whether the intended completion is validation, a warning, or documentation that prevents the silent increase.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.