Provide an HTTP Sink plugin

Is your feature request related to a problem? Please describe.

Data Prepper can output to OpenSearch, but cannot output to arbitrary HTTP endpoints.

Describe the solution you'd like

Create a new sink in Data Prepper which outputs JSON to an arbitrary HTTP endpoint.

It can output events to an HTTP endpoint as a JSON array of events.

Example:

[{"key1": "value1", "key2": "value2"}, {"key1": "value3", "key2": "value4"}]

It could also be configured to write each new event as a JSON string on its own line. This is similar to the OpenSearch bulk API.

Example:

{"key1": "value1", "key2": "value2"}
{"key1": "value3", "key2": "value4"}

Additional Context

This sink should be able to write to the existing http source plugin and its existing JsonCodec.

Possible user configurations

• url - The URL to send events to
• proxy - A proxy server to send requests through
• http_method - Which HTTP method to use. Defaults to POST.
• format - Configuration to determine the format of the data. May be json-array for a JSON array, or json-nd for JSON-ND.
• Pluggable authentication options. Should support HTTP Basic, Bearer token, and mTLS to start
• Configurations to specify the SSL certificates or chains to trust (File, S3 object, ACM). This can be similar to the options provided in the opensearch sink.
• SSL verification modes similar to core peer forwarding.
• Options to control retrying failed requests (which status codes; how many retries; backoff approach)
• Timeout options (connect, request, socket)

Start by reviewing the existing http source plugin's JsonCodec and the opensearch sink for comparable behavior. Also read PeerForwarderConfiguration for SSL verification options. Done should be a new sink that sends events to arbitrary HTTP endpoints in JSON-array or JSON-ND format, with the listed authentication, SSL, retry, proxy, method, and timeout configurations.