Graylog2 / Graylog2/graylog-plugin-aws

Handle cloudtrail message where the message is not json

Open
#249 4 comments 0 reactions 0 assignees View on GitHub
bug triaged
Dominant language
Java
Stars
93
Forks
37
PR merge metrics
No merged PRs in 30d

Description

Stacktrace

```
2019-09-24T15:42:47.206Z ERROR [CloudtrailSNSNotificationParser] Parsing exception.
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'CloudTrail': was expecting ('true', 'false' or 'null')
at [Source: CloudTrail validation message.; line: 1, column: 11]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1702) ~[graylog.jar:?]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:558) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2839) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1903) ~[graylog.jar:?]
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:749) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3850) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3799) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2858) ~[graylog.jar:?]
at org.graylog.aws.inputs.cloudtrail.notifications.CloudtrailSNSNotificationParser.parse(CloudtrailSNSNotificationParser.java:36) [graylog-plugin-aws-3.1.2.jar:?]
at org.graylog.aws.inputs.cloudtrail.notifications.CloudtrailSQSClient.getNotifications(CloudtrailSQSClient.java:55) [graylog-plugin-aws-3.1.2.jar:?]
at org.graylog.aws.inputs.cloudtrail.CloudTrailSubscriber.run(CloudTrailSubscriber.java:89) [graylog-plugin-aws-3.1.2.jar:?]
```

This is caused by those messages being put in the SQS queue by SNS sometimes (not sure what causes it). The Message field is not JSON so it fails to be parsed and the message stays in the queue and gets refetched forever in a loop, polluting the graylog logs with the stacktrace.

```js
{
"Type" : "Notification",
"MessageId" : "xxxxxx",
"TopicArn" : "arn:aws:sns:us-east-1:xxxxxxxx:cloudtrail-logs-delivery-logs",
"Message" : "CloudTrail validation message.",
"Timestamp" : "2019-09-24T14:51:30.832Z",
"SignatureVersion" : "1",
"Signature" : "xxxxxx",
"SigningCertURL" : "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-6aad65c2f9911b05cd53efda11f913f9.pem",
"UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:xxxxx:cloudtrail-logs-delivery-logs:xxxx"
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.