Azure / Azure/azure-linux-extensions

FileLogs does not write in SA

Open
#516 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
333
Forks
278
Avg merge
2d 9h
Merged PRs (30d)
4

Description

I configured arm template for VMSS and VM, on VMSS it works. but on VM don't, I search in logs but find nothing conclusive.
The token I tried with and without question-mark '?'.
The LAD gets the metrics writes on SA, but don't get filelog (neither blobs or tables, however it creates the container).
OS is CentOS 7.4 for all vms

So here's the config file:
"properties": {
"publisher": "Microsoft.Azure.Diagnostics",
"type": "LinuxDiagnostic",
"typeHandlerVersion": "3.0",
"autoUpgradeMinorVersion": true,
"settings": {
"StorageAccount": "[variables('saDiagName')]",
"sampleRateInSeconds": 15,
"ladCfg": {
"diagnosticMonitorConfiguration": {
"performanceCounters": {
"sinks": "WADMetricJsonBlob",
"performanceCounterConfiguration": [
{
"unit": "Percent",
"type": "builtin",
"counter": "PercentProcessorTime",
"counterSpecifier": "/builtin/Processor/PercentProcessorTime",
"annotation": [
{
"locale": "en-us",
"displayName": "Aggregate CPU %utilization"
}
],
"condition": "IsAggregate=TRUE",
"class": "Processor"
},
{
"unit": "Bytes",
"type": "builtin",
"counter": "UsedSpace",
"counterSpecifier": "/builtin/FileSystem/UsedSpace",
"annotation": [
{
"locale": "en-us",
"displayName": "Used disk space on /"
}
],
"condition": "Name=\"/\"",
"class": "Filesystem"
}
]
},
"metrics": {
"metricAggregation": [
{
"scheduledTransferPeriod": "PT1H"
},
{
"scheduledTransferPeriod": "PT1M"
}
],
"resourceId": "[concat(variables('ladMetricsResourceId'),variables('vmName'))]"
},
"eventVolume": "Large",
"syslogEvents": {
"sinks": "SyslogJsonBlob",
"syslogEventConfiguration": {
"LOG_USER": "LOG_INFO"
}
}
},
"fileLogs": [
{
"file": "/.../log.log",
"table":"myCustomLog",
"sinks": "myCustomLog"
},
{
"file": "/var/log/waagent.log",
"table":"filelogJsonBlob",
"sinks": "filelogJsonBlob"
},
{
"file": "/var/log/messages",
"table":"messagesJsonBlob",
"sinks": "messagesJsonBlob"
}
]
}
},
"protectedSettings": {
"storageAccountName": "[variables('saDiagName')]",
"storageAccountSasToken": "[parameters('token')]",
"sinksConfig": {
"sink": [
{
"name": "WADMetricJsonBlob",
"type": "JsonBlob"
},
{
"name": "SyslogJsonBlob",
"type": "JsonBlob"
},
{
"name": "myCustomLog",
"type": "JsonBlob"
},
{
"name": "filelogJsonBlob",
"type": "JsonBlob"
},
{
"name": "messagesJsonBlob",
"type": "JsonBlob"
}
]
}
}

the extension.log:
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] sequence number is 0
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] setting file path is/var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/config/0.settings
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] JSON config: {"runtimeSettings": [{"handlerSettings": {"protectedSettings": "...", "publicSettings": {"ladCfg": ...}
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Config decoded correctly.
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Disable verbose log
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Dispatching command:-enable
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] return not alive cat: /proc/1899/cmdline: No such file or directory
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] get pids:[]
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] return not alive cat: /proc/1899/cmdline: No such file or directory
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] start daemon ['python', '/var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/diagnostic.py', '-daemon']
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] return not alive cat: /proc/1899/cmdline: No such file or directory
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] sequence number is 0
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] setting file path is/var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/config/0.settings
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] JSON config: ...
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Config decoded correctly.
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Disable verbose log
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Dispatching command:-daemon
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Deployment ID found: 99ef70ab-07ec-4495-b206-357c0ee841a4.
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] LAD settings with secrets redacted: ...
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Event volume found in ladCfg: Large
2018/01/23 19:19:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Run with timeout: rpm -qi policycoreutils-python; if [ ! $? == 0 ]; then yum install -y policycoreutils-python; fi
2018/01/23 19:19:52 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return None
2018/01/23 19:19:52 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Name : policycoreutils-python
...
2018/01/23 19:19:52 Build Date : Fri 04 Aug 2017 09:00:38 AM UTC
..
2018/01/23 19:19:52 Description :
2018/01/23 19:19:52 The policycoreutils-python package contains the management tools use to manage
2018/01/23 19:19:52 an SELinux environment.
2018/01/23 19:19:52
2018/01/23 19:19:53 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Begin omsagent setup.
2018/01/23 19:19:53 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd sh omsagent-*.universal.x64.sh --upgrade
2018/01/23 19:19:56 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Enable,success,0,Enable succeeded
2018/01/23 19:21:47 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:Checking host architecture ...
2018/01/23 19:21:47 Extracting...
2018/01/23 19:21:47 Updating OMS agent ...
2018/01/23 19:21:47 ----- Checking package: omsagent (omsagent-1.4.0-45.universal.x64) -----
2018/01/23 19:21:47 ----- Checking package: omsconfig (omsconfig-1.1.1-559.x64) -----
2018/01/23 19:21:47 Extracting...
2018/01/23 19:21:47 Updating cross-platform agent ...
2018/01/23 19:21:47 ----- Upgrading package: omi (omi-1.2.0-35.ulinux.x64) -----
2018/01/23 19:21:47 Creating omiusers group ...
2018/01/23 19:21:47
2018/01/23 19:21:47 ************************************************************
2018/01/23 19:21:47 * Warning: The certificate and keyfile were not generated *
2018/01/23 19:21:47 * since they already exist. *
2018/01/23 19:21:47 ************************************************************
2018/01/23 19:21:47 Configuring OMI service ...
2018/01/23 19:21:47 Created symlink from /etc/systemd/system/multi-user.target.wants/omid.service to /usr/lib/systemd/system/omid.service.
2018/01/23 19:21:47 ----- Upgrading package: scx (scx-1.6.3-274.universal.x64) -----
2018/01/23 19:21:47 ----- Updating bundled packages -----
2018/01/23 19:21:47 Checking if Apache is installed ...
2018/01/23 19:21:47 Apache not found, will not install
2018/01/23 19:21:47 Checking if MySQL is installed ...
2018/01/23 19:21:47 MySQL not found, will not install
2018/01/23 19:21:47 Preparing... ########################################
2018/01/23 19:21:47 Creating omsagent group ...
2018/01/23 19:21:47 Creating omsagent service account ...
2018/01/23 19:21:47 Creating nxautomation group ...
2018/01/23 19:21:47 Creating nxautomation service account ...
2018/01/23 19:21:47 Updating / installing...
2018/01/23 19:21:47 omsagent-1.4.0-45 ########################################
2018/01/23 19:21:47 grep: /etc/opt/microsoft/omsagent/conf/omsadmin.conf: No such file or directory
2018/01/23 19:21:47 System appears to have SELinux installed, attempting to install selinux policy module for logrotate
2018/01/23 19:21:47 Trying /usr/share/selinux/packages/omsagent-logrotate/omsagent-logrotate.pp ...
2018/01/23 19:21:47 Configuring OMS agent service ...
2018/01/23 19:21:47 Checking for ctypes python module...
2018/01/23 19:21:47 omsconfig-1.1.1-559 ########################################
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 Extracting to /opt/microsoft/omsconfig/modules
2018/01/23 19:21:47 Installing resource MSFT_nxGroupResource
2018/01/23 19:21:47 Installing resource MSFT_nxPackageResource
2018/01/23 19:21:47 Installing resource MSFT_nxServiceResource
2018/01/23 19:21:47 Installing resource MSFT_nxAvailableUpdatesResource
2018/01/23 19:21:47 Installing resource MSFT_nxUserResource
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 Extracting to /opt/microsoft/omsconfig/modules
2018/01/23 19:21:47 Installing resource MSFT_nxOMSPerfCounterResource
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 Extracting to /opt/microsoft/omsconfig/modules
2018/01/23 19:21:47 Installing resource MSFT_nxOMSSyslogResource
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 Extracting to /opt/microsoft/omsconfig/modules
2018/01/23 19:21:47 Installing resource MSFT_nxOMSKeyMgmtResource
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 Extracting to /opt/microsoft/omsconfig/modules
2018/01/23 19:21:47 Installing resource MSFT_nxOMSCustomLogResource
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 Extracting to /opt/microsoft/omsconfig/modules
2018/01/23 19:21:47 Installing resource MSFT_nxOMSSudoCustomLogResource
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 Extracting to /opt/microsoft/omsconfig/modules
2018/01/23 19:21:47 Installing resource MSFT_nxFileInventoryResource
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 gpg: key ...:"Microsoft (Release Signing) " not changed
2018/01/23 19:21:47 gpg: Total number processed: 1
2018/01/23 19:21:47 gpg: unchanged: 1
2018/01/23 19:21:47 su: warning: cannot change directory to /var/opt/microsoft/omsagent/run: No such file or directory
2018/01/23 19:21:47 gpg: key ...:"Microsoft (Release Signing) " not changed
2018/01/23 19:21:47 gpg: Total number processed: 1
2018/01/23 19:21:47 gpg: unchanged: 1
2018/01/23 19:21:47 ----- Updating bundled packages -----
2018/01/23 19:21:47 Checking if Docker is installed...
2018/01/23 19:21:47 Docker not found. Docker agent will not be installed.
2018/01/23 19:21:47 Checking if required dependencies for auoms are installed...
2018/01/23 19:21:47 Extracting...
2018/01/23 19:21:47 Updating auoms ...
2018/01/23 19:21:47 ----- Updating package: auoms (auoms-1.0.0-2.universal.x64) -----
2018/01/23 19:21:47 System appears to have SELinux installed, attempting to install selinux policy module for auoms
2018/01/23 19:21:47 Installing auoms selinux policy version 1.0
2018/01/23 19:21:47 Trying /usr/share/selinux/packages/auoms/auoms.pp
2018/01/23 19:21:47 Labeling auoms files
2018/01/23 19:21:47 /sbin/restorecon reset /opt/microsoft/auoms/bin/auoms context system_u:object_r:bin_t:s0->system_u:object_r:auoms_exec_t:s0
2018/01/23 19:21:47 Restarting auditd
2018/01/23 19:21:47 Stopping logging: [ OK ]^M
2018/01/23 19:21:47 Redirecting start to /bin/systemctl start auditd.service
2018/01/23 19:21:47 Shell bundle exiting with code 0
2018/01/23 19:21:47
2018/01/23 19:21:48 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd sed -i "s/RestartService syslog-ng\\s*$/RestartService syslog-ng || RestartService syslog/g" /opt/microsoft/omsagent/bin/configure_syslog.sh
2018/01/23 19:21:48 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:
2018/01/23 19:21:48 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd /opt/microsoft/omsagent/ruby/bin/fluent-gem uninstall fluent-plugin-mdsd -a
2018/01/23 19:21:48 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:Successfully uninstalled fluent-plugin-mdsd-0.1.3.pre.build.develop.46
2018/01/23 19:21:48
2018/01/23 19:21:48 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd /opt/microsoft/omsagent/ruby/bin/fluent-gem install fluent-plugin-mdsd-*.gem
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:Successfully installed fluent-plugin-mdsd-0.1.3.pre.build.develop.46
2018/01/23 19:21:50 Parsing documentation for fluent-plugin-mdsd-0.1.3.pre.build.develop.46
2018/01/23 19:21:50 Installing ri documentation for fluent-plugin-mdsd-0.1.3.pre.build.develop.46
2018/01/23 19:21:50 Done installing documentation for fluent-plugin-mdsd after 0 seconds
2018/01/23 19:21:50 1 gem installed
2018/01/23 19:21:50
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd grep '^\s*httpsport\s*=' /etc/opt/omi/conf/omiserver.conf | grep -v '^\s*httpsport\s*=\s*0\s*$'
2018/01/23 19:21:50 ERROR:CalledProcessError. Error Code is 1
2018/01/23 19:21:50 ERROR:CalledProcessError. Command string was grep '^\s*httpsport\s*=' /etc/opt/omi/conf/omiserver.conf | grep -v '^\s*httpsport\s*=\s*0\s*$'
2018/01/23 19:21:50 ERROR:CalledProcessError. Command result was
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 1:
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd sh /opt/microsoft/omsagent/bin/configure_syslog.sh unconfigure LAD 25224
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd sh /opt/microsoft/omsagent/bin/configure_syslog.sh configure LAD 25224
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:Configuring rsyslog for OMS logging
2018/01/23 19:21:50 Restarting service: rsyslog
2018/01/23 19:21:50
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd sh /opt/microsoft/omsagent/bin/configure_syslog.sh restart LAD
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:Restarting service: rsyslog
2018/01/23 19:21:50
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd /opt/microsoft/omsagent/bin/service_control restart LAD
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd rpm -i --force lad-mdsd-*.rpm
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:
2018/01/23 19:21:50 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd SSL_CERT_FILE=/etc/pki/tls/cert.pem SSL_CERT_DIR=/etc/pki/tls/certs /usr/local/lad/bin/mdsd -v -c /var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/xmlCfg.xml
2018/01/23 19:21:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:Parse succeeded with no messages.
2018/01/23 19:21:51
2018/01/23 19:21:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd /opt/omi/bin/service_control is-running
2018/01/23 19:21:51 ERROR:CalledProcessError. Error Code is 1
2018/01/23 19:21:51 ERROR:CalledProcessError. Command string was /opt/omi/bin/service_control is-running
2018/01/23 19:21:51 ERROR:CalledProcessError. Command result was
2018/01/23 19:21:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 1:
2018/01/23 19:21:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd checkmodule -M -m -o /var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/lad_mdsd.mod /var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/lad_mdsd.te
2018/01/23 19:21:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:checkmodule: loading policy configuration from /var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/lad_mdsd.te
2018/01/23 19:21:51 checkmodule: policy configuration loaded
2018/01/23 19:21:51 checkmodule: writing binary representation (version 17) to /var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/lad_mdsd.mod
2018/01/23 19:21:51
2018/01/23 19:21:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd semodule_package -o /var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/lad_mdsd.pp -m /var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/lad_mdsd.mod
2018/01/23 19:21:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:
2018/01/23 19:21:51 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd semodule -u /var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/lad_mdsd.pp
2018/01/23 19:22:08 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:The --upgrade option is deprecated. Use --install instead.
2018/01/23 19:22:08
2018/01/23 19:22:08 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] RunCmd rm -f /var/run/mdsd/lad_mdsd.pidport
2018/01/23 19:22:08 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Return 0:
2018/01/23 19:22:08 [Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109] Start mdsd ['/usr/local/lad/bin/mdsd', '-A', '-C', '-c', '/var/lib/waagent/Microsoft.Azure.Diagnostics.LinuxDiagnostic-3.0.109/xmlCfg.xml', '-R', '-r', 'lad_mdsd', '-e', '/var/log/azure/Microsoft.Azure.Diagnostics.LinuxDiagnostic/3.0.109/mdsd.err', '-w', '/var/log/azure/Microsoft.Azure.Diagnostics.LinuxDiagnostic/3.0.109/mdsd.warn', '-o', '/var/log/azure/Microsoft.Azure.Diagnostics.LinuxDiagnostic/3.0.109/mdsd.info']

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.