awslabs / awslabs/lambda-streams-to-firehose
Problem with a Cloudwatch Logs Destination to Kinesis stream?
- Dominant language
- JavaScript
- Stars
- 278
- Forks
- 90
- PR merge metrics
- No merged PRs in 30d
Description
I am using a CWL Destination sending to a Stream and then using your Lambda to send it on to Firehose - S3, no Firehose compression or encryption.
The files that show up look like some strange unicode format. Is there an issue with using a Destination as a CWL subscription to Kinesis stream here?
Example file contents in S3:
```
"\u001f�\b\u0000\u0000\u0000\u0000\u0000\u0000\u00005��\n�@\u0014Ee�uD���\u000b1\u0017YB\n-\"bҗ>�\u0019�7\u0016\u0011�{c��p/��7��H\u0014��\u001a�\u001e��}z���.H�u\u0018�\u0011WO\t�O*��Oa�2R\u0005٠RE�U��l��h\u0010���^)��\u0018Tr��\u0001M�;����\u0001�����\u000fu�VÈڎM\u001dw>sW�r�p��_�\u00178F���~z\u001e�K��(\u000bV��L�ԍ�v\t����\u0016%\u0010\u0012��ژw��\u0003_��\u0003�\u0000\u0000\u0000"
```
Contributor guide
Research direction
The issue names no repository file or test. Start by reproducing the CloudWatch Logs destination to Kinesis, Lambda, Firehose, and S3 path, then inspect the Lambda forwarding entry point and compare the resulting bytes with the example S3 content. Done means identifying whether the forwarded payload is being handled incorrectly and recording or fixing the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, javascript
- Domain
- cloud, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100