Graylog2 / Graylog2/graylog-plugin-aws
Handle cloudtrail message where the message is not json
- Dominant language
- Java
- Stars
- 93
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
Stacktrace
```
2019-09-24T15:42:47.206Z ERROR [CloudtrailSNSNotificationParser] Parsing exception.
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'CloudTrail': was expecting ('true', 'false' or 'null')
at [Source: CloudTrail validation message.; line: 1, column: 11]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1702) ~[graylog.jar:?]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:558) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2839) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1903) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:749) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3850) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3799) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2858) ~[graylog.jar:?]
at org.graylog.aws.inputs.cloudtrail.notifications.CloudtrailSNSNotificationParser.parse(CloudtrailSNSNotificationParser.java:36) [graylog-plugin-aws-3.1.2.jar:?]
at org.graylog.aws.inputs.cloudtrail.notifications.CloudtrailSQSClient.getNotifications(CloudtrailSQSClient.java:55) [graylog-plugin-aws-3.1.2.jar:?]
at org.graylog.aws.inputs.cloudtrail.CloudTrailSubscriber.run(CloudTrailSubscriber.java:89) [graylog-plugin-aws-3.1.2.jar:?]
```
This is caused by those messages being put in the SQS queue by SNS sometimes (not sure what causes it). The Message field is not JSON so it fails to be parsed and the message stays in the queue and gets refetched forever in a loop, polluting the graylog logs with the stacktrace.
```js
{
"Type" : "Notification",
"MessageId" : "xxxxxx",
"TopicArn" : "arn:aws:sns:us-east-1:xxxxxxxx:cloudtrail-logs-delivery-logs",
"Message" : "CloudTrail validation message.",
"Timestamp" : "2019-09-24T14:51:30.832Z",
"SignatureVersion" : "1",
"Signature" : "xxxxxx",
"SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-6aad65c2f9911b05cd53efda11f913f9.pem",
"UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:xxxxx:cloudtrail-logs-delivery-logs:xxxx"
}
```
Contributor guide
Assessment
This issue has not been assessed yet.