Azure / Azure/azure-quickstart-templates

Splunk-On-Ubuntu fails due to script / VM Extension issue

Open
#5,926 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Bicep
Stars
14.9k
Forks
16.2k
Avg merge
6d 21h
Merged PRs (30d)
6

Description

Template Name
"Splunk-on-ubuntu"
Template link:
https://github.com/Azure/azure-quickstart-templates/tree/master/splunk-on-ubuntu

### Issue Details
Cluster deployment fails repeatedly when clicking "deploying to Azure" from the GitHub Repository and entering parameters in the Azure Portal. I am Owner of Subscription and so no RBAC issues should be seen.

The deployment fails complaining about the Scripts (VM Extensions):

{ "id": "/subscriptions/496c0836-f2ce-495b-a613-21bbfc120589/resourceGroups/splunk-on-ubuntu2/providers/Microsoft.Resources/deployments/clustermaster/operations/24AF5A63B7AFD427", "operationId": "24AF5A63B7AFD427", "properties": { "provisioningOperation": "Create", "provisioningState": **"Failed**", "timestamp": "2019-03-28T15:28:28.873662Z", "duration": "PT3M0.7030184S", "trackingId": "5e8cd0b5-2dc0-4951-b69e-0a6a7902da20", "serviceRequestId": "c63c1525-2c48-4ffb-a1ab-d2523c0db104", "statusCode": "Conflict", "statusMessage": { "status": "Failed", "error": { "code": "ResourceDeploymentFailure", "message": "The resource operation completed with terminal provisioning state 'Failed'.", "details": [ { "code": "**VMExtensionProvisioningError**", "message": "VM has reported a failure when processing extension 'node-setup'. Error message: \"Enable failed: failed to execute command: command terminated with exit status=1\n[stdout]\nexecute(\"update-splunk-mgmt-port\") do\n\u001b[0m action [:run]\n\u001b[0m default_guard_interpreter :execute\n\u001b[0m command \"/opt/splunk/bin/splunk set splunkd-port 8089 -auth 'admin:Cisco1234567890'\"\n\u001b[0m backup 5\n\u001b[0m environment {\"HOME\"=>\"/opt/splunk\", \"USER\"=>\"splunk\"}\n\u001b[0m group \"splunk\"\n\u001b[0m returns 0\n\u001b[0m user \"splunk\"\n\u001b[0m declared_type :execute\n\u001b[0m cookbook_name \"chef-splunk\"\n\u001b[0m recipe_name \"server\"\n\u001b[0m domain nil\n\u001b[0m not_if \"/opt/splunk/bin/splunk show splunkd-port -auth 'admin:Cisco1234567890' | grep ': 8089'\"\n\u001b[0m end\n\u001b[0m \n\u001b[0m System Info:\u001b[0m\n ------------\u001b[0m\n chef_version=13.0.118\n\u001b[0m platform=ubuntu\n\u001b[0m platform_version=14.04\n\u001b[0m ruby=ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]\n\u001b[0m program_name=chef-client worker: ppid=2397;start=15:26:36;\n\u001b[0m executable=/opt/chef/bin/chef-client\u001b[0m\n \n\u001b[0m[2019-03-28T15:28:22+00:00] INFO: Running queued delayed notifications before re-raising exception\n[2019-03-28T15:28:22+00:00] INFO: execute[update-datastore-dir] sending restart action to service[splunk] (delayed)\nRecipe: chef-splunk::service\u001b[0m\n * service[splunk] action restart[2019-03-28T15:28:22+00:00] INFO: Processing service[splunk] action restart (chef-splunk::service line 104)\n\n \u001b[0m\n ================================================================================\u001b[0m\n \u001b[31mError executing action `restart` on resource 'service[splunk]'\u001b[0m\n ================================================================================\u001b[0m\n \n\u001b[0m Mixlib::ShellOut::ShellCommandFailed\u001b[0m\n ------------------------------------\u001b[0m\n Expected process to exit with [0], but received '8'\n\u001b[0m ---- Begin output of /etc/init.d/splunk restart ----\n\u001b[0m STDOUT: Restarting Splunk...\n\u001b[0m ERROR: Couldn't determine $SPLUNK_HOME or $SPLUNK_ETC; perhaps one should be set in environment\n\u001b[0m STDERR: \n\u001b[0m ---- End output of /etc/init.d/splunk restart ----\n\u001b[0m Ran /etc/init.d/splunk restart returned 8\u001b[0m\n \n\u001b[0m Resource Declaration:\u001b[0m\n ---------------------\u001b[0m\n # In /etc/chef/local-mode-cache/cache/cookbooks/chef-splunk/recipes/service.rb\n\u001b[0m \n\u001b[0m 104: service 'splunk' do\n\u001b[0m 105: supports status: true, restart: true\n\u001b[0m 106: provider Chef::Provider::Service::Init\n\u001b[0m 107: action :start\n\u001b[0m 108: end\n\u001b[0m 109: end\n\u001b[0m \n\u001b[0m Compiled Resource:\u001b[0m\n ------------------\u001b[0m\n # Declared in /etc/chef/local-mode-cache/cache/cookbooks/chef-splunk/recipes/service.rb:104:in `from_file'\n\u001b[0m \n\u001b[0m service(\"splunk\") do\n\u001b[0m provider Chef::Provider::Service::Init\n\u001b[0m action [:start]\n\u001b[0m updated true\n\u001b[0m default_guard_interpreter :default\n\u001b[0m service_name \"splunk\"\n\u001b[0m enabled nil\n\u001b[0m running false\n\u001b[0m masked nil\n\u001b[0m pattern \"splunk\"\n\u001b[0m declared_type :service\n\u001b[0m cookbook_name \"chef-splunk\"\n\u001b[0m recipe_name \"service\"\n\u001b[0m supports {:status=>true, :restart=>true}\n\u001b[0m end\n\u001b[0m \n\u001b[0m System Info:\u001b[0m\n ------------\u001b[0m\n chef_version=13.0.118\n\u001b[0m platform=ubuntu\n\u001b[0m platform_version=14.04\n\u001b[0m ruby=ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]\n\u001b[0m program_name=chef-client worker: ppid=2397;start=15:26:36;\n\u001b[0m executable=/opt/chef/bin/chef-client\u001b[0m\n \n\u001b[0m\u001b[0m\nRunning handlers:\u001b[0m\n[2019-03-28T15:28:22+00:00] ERROR: Running exception handlers\nRunning handlers complete\n\u001b[0m[2019-03-28T15:28:22+00:00] ERROR: Exception handlers complete\nChef Client failed. 14 resources updated in 01 minutes 45 seconds\u001b[0m\n[2019-03-28T15:28:22+00:00] FATAL: Stacktrace dumped to /etc/chef/local-mode-cache/cache/chef-stacktrace.out\n[2019-03-28T15:28:22+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report\n[2019-03-28T15:28:22+00:00] ERROR: Chef::Exceptions::MultipleFailures\n[2019-03-28T15:28:22+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)\n\n[stderr]\nDevice contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel\nBuilding a new DOS disklabel with disk identifier 0x256f88ac.\nChanges will remain in memory only, until you decide to write them.\nAfter that, of course, the previous content won't be recoverable.\n\nWarning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)\nDevice contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel\nBuilding a new DOS disklabel with disk identifier 0x4c9864da.\nChanges will remain in memory only, until you decide to write them.\nAfter that, of course, the previous content won't be recoverable.\n\nWarning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)\nmdadm: Defaulting to version 1.2 metadata\nmdadm: array /dev/md0 started.\nmke2fs 1.42.9 (4-Feb-2014)\n\"." } ] } }, "targetResource": { "id": "/subscriptions/496c0836-f2ce-495b-a613-21bbfc120589/resourceGroups/splunk-on-ubuntu2/providers/Microsoft.Compute/virtualMachines/cm-vm/extensions/node-setup", "resourceType": "Microsoft.Compute/virtualMachines/extensions", "resourceName": "cm-vm/node-setup" } }}

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the splunk-on-ubuntu template and reproduce the deployment failure at the node-setup VM extension. Read the reported chef-splunk service.rb resource around line 104 and inspect the extension output, including the Splunk restart and storage setup errors. Done means the template deploys successfully without the VM extension failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
ubuntu
Domain
cloud, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.