Azure / Azure/azure-quickstart-templates
application-gateway-logviewer-goaccess - Error Parsing Log Entries
- Dominant language
- Bicep
- Stars
- 14.9k
- Forks
- 16.2k
- Avg merge
- 6d 21h
- Merged PRs (30d)
- 6
Description
[application-gateway-logviewer-goaccess](https://github.com/Azure/azure-quickstart-templates/tree/master/application-gateway-logviewer-goaccess)
### Issue
It seems like the template deployed, but no report.html was produced. I navigated to `/var/log/azure/Microsoft.Azure.Networking.ApplicationGateway.LogProcessor/application.log` where it looks like there is an error every time that the `Microsoft.Azure.Networking.ApplicationGateway.LogProcessor` attempts to Parse lines from the log file.
It appears that the Processor is able to go and fetch the files successfully, considering it is having problems parsing real data. I assume that there must be an issue with the custom log format developed in the .zip? My WAF is Azure V2.
### Error Example
> `2020-11-25 15:21:41,512 INFO [6] ?.? - ProcessBlobCache: BlobContext: Microsoft.Azure.Networking.ApplicationGateway.LogProcessor.CloudBlobContext
2020-11-25 15:21:41,513 INFO [5] ?.? - PostBlobData: Starting Processing File: REMOVED
2020-11-25 15:21:41,568 ERROR [5] ?.? - PostBlobData: Error while processing line : { "timeStamp": "2020-11-25T15:00:04+00:00", "resourceId": "/SUBSCRIPTIONS/sanitized/RESOURCEGROUPS/MPPCCORENETRG/PROVIDERS/MICROSOFT.NETWORK/APPLICATIONGATEWAYS/MPPCWAFGW", "listenerName": "sanitized", "ruleName": "sanitized", "backendPoolName": "sanitized", "backendSettingName": "sanitized", "operationName": "ApplicationGatewayAccess", "category": "ApplicationGatewayAccessLog", "properties": {"instanceId":"appgw_3","clientIP":"sanitized","clientPort":sanitized,"httpMethod":"GET","originalRequestUriWithArgs":"\/available","requestUri":"\/available","requestQuery":"","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/87.0.4280.66 Safari\/537.36","httpStatus":200,"httpVersion":"HTTP\/2.0","receivedBytes":300,"sentBytes":663,"timeTaken":0.008,"transactionId":"sanitized","sslEnabled":"on","sslCipher":"ECDHE-RSA-AES128-GCM-SHA256","sslProtocol":"TLSv1.2","sslClientVerify":"NONE","sslClientCertificateFingerprint":"","sslClientCertificateIssuerName":"","serverRouted":"sanitized:443","serverStatus":"200","serverResponseLatency":"0.008","originalHost":"sanitized","host":"sanitized"}}
2020-11-25 15:21:41,575 ERROR [5] ?.? - PostBlobData: Caught Exception: System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Azure.Networking.ApplicationGateway.LogProcessor.CustomLogProcessor.ProcessLog(String logLine, Boolean& transactionComplete) in E:\src\networkmonitoring\Tools\src\AppGatewayLogProcessor\Services\CustomLogProcessor.cs:line 88
at Microsoft.Azure.Networking.ApplicationGateway.LogProcessor.Program.PostBlobData(Object state) in E:\src\networkmonitoring\Tools\src\AppGatewayLogProcessor\Program.cs:line 251`
### Reproduce:
I think that if you deploy the template yourself using the instructions provided (be specific in the SAS token section) you will encounter the same issue, seeing as there is nothing special with my WAF.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the deployed application.log and the application-gateway-logviewer-goaccess template instructions, then inspect CustomLogProcessor.cs at line 88 and Program.cs at line 251 from the reported stack trace. Reproduce with an Azure WAF V2 deployment and determine why the shown access-log line fails parsing; done means processing completes and report.html is produced without the logged exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- cloud, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100