New `aws_cloudwatch_logs` source

The goal of this source is to be a simple way to start consuming logs from AWS in low volume environments by pulling logs from exposed AWS CloudWatch APIs (FilterLogEvents and GetLogEvents).

There will be a relatively low volume limit to this source (< 5 MBps / 50k events in the case of FilterLogEvents) which we should document and take extra care around surfacing when we are hitting AWS rate limits.

We've also observed that FilterLogEvents does not perform well for groups with a large number of streams. If we go with this API, we should document this restriction (and provide knobs for specificity specific streams or stream prefix).

I'd probably recommend an RFC just for this source to suss out any decision points and to provide the answers to open questions, including:

• Do we want to allow users to scrape to scrape multiple log groups with a single source? Do we want to allow wildcards for the group name if so?
• Decide on FilterLogEvents vs. GetLogEvents (or do we want to support both?)
• Do we want to allow consuming a log group from its beginning? Or just newly published events?
• Do we want to persist checkpoints on disk to allow for restarts? Or just always start from "now"?
• If we go with FilterLogEvents do we want to allow users to specify a filter?
• Should we rely on the AWS pagination token (nextToken) for checkpoints or store our own serialized cursor that we can use to pick back up. nextToken is only valid for 24 hours and, I think, is tied to the filter pattern for FilterLogEvents (though we can verify)

Examples of other implementations:

• https://github.com/fluent-plugins-nursery/fluent-plugin-cloudwatch-logs
• https://github.com/lukewaite/logstash-input-cloudwatch-logs

External refs:

• https://github.com/aws/containers-roadmap/issues/66

Start by reviewing the AWS CloudWatch Logs APIs for FilterLogEvents and GetLogEvents, then compare the linked Fluentd and Logstash implementations. Resolve the listed RFC questions around log groups, API choice, starting position, checkpoints, filtering, pagination, and rate limits. Done means an agreed source design and documented restrictions.