Support CloudWatch Logs as a Sink

Problem Satement:

• Right now, there is no support for sending log information to CloudWatch Logs (CWL). Making it difficult for users of Data-Prepper(DP) library that want to take advantage of these features. Such as log monitoring, querying and alarming. In order to extend the flexibility of DP to reach the users of CWL, DP would have to extend their sink functionality to reaching the CWL service endpoint.

Solution:

• A solution to this would be the addition of a DP Sink as a plugin. This plugin would function like any of the other sinks with its own configuration parameters that can be tuned to fit the use. This plugin should integrate seamlessly and continue to work with the event structure that DP follows.

The CWL-Sink should support:

• The handling of AWS credentials for account verification and R/W access.
• The Log-Group endpoint to which DP should publish logs to. (This would be defined in the yaml file)
• The Sink should buffer logs to provide reliable data transfer. (in the meantime should support JSON)