awslabs / awslabs/lambda-streams-to-firehose

Problem with a Cloudwatch Logs Destination to Kinesis stream?

Open
#35 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
278
Forks
90
PR merge metrics
No merged PRs in 30d

Description

I am using a CWL Destination sending to a Stream and then using your Lambda to send it on to Firehose - S3, no Firehose compression or encryption.
The files that show up look like some strange unicode format. Is there an issue with using a Destination as a CWL subscription to Kinesis stream here?

Example file contents in S3:
```
"\u001f�\b\u0000\u0000\u0000\u0000\u0000\u0000\u00005��\n�@\u0014Ee�uD���\u000b1\u0017YB\n-\"bҗ>�\u0019�7\u0016\u0011�{c��p/��7��H\u0014��\u001a�\u001e��}z���.H�u\u0018�\u0011WO\t�O*��Oa�2R\u0005٠RE�U��l��h\u0010���^)��\u0018Tr��\u0001M�;����\u0001�����\u000fu�VÈڎM\u001dw>sW�r�p��_�\u00178F���~z\u001e�K��(\u000bV��L�ԍ�v\t����\u0016%\u0010\u0012��ژw��\u0003_��\u0003�\u0000\u0000\u0000"
```

Contributor guide

Open the contributing guide

Research direction

The issue names no repository file or test. Start by reproducing the CloudWatch Logs destination to Kinesis, Lambda, Firehose, and S3 path, then inspect the Lambda forwarding entry point and compare the resulting bytes with the example S3 content. Done means identifying whether the forwarded payload is being handled incorrectly and recording or fixing the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, javascript
Domain
cloud, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.