[aws-for-fluent-bit] kinesis_streams Does not support compression and aggregation
- Dominant language
- Mustache
- Stars
- 1.3k
- Forks
- 1.1k
- Avg merge
- 22m
- Merged PRs (30d)
- 5
Description
Hi Team,
While using the below config I'm getting error that aggregation and compression is not supported.
I'm using `chart_revision: 0.1.35`
```
kinesis_streams:
enabled: true
match: '*'
region: us-east-1
stream: input-stream
role_arn: "arn:aws:iam::xxxxx:role/writer-role"
time_key: eventTime
time_key_format: "%Y-%m-%dT%H:%M:%S.%9NZ"
sts_endpoint: "https://sts.us-east-1.amazonaws.com"
partition_key: PROGRAM
extraOutputs: |
aggregation true
compression zlib
```
## Error
```
[error] [config] kinesis_streams: unknown configuration property 'compression'.
The following properties are allowed: region, stream, time_key, time_key_format, role_arn, endpoint, sts_endpoint, external_id, log_key, and auto_retry_requests.
[2025/03/21 18:43:31] [error] [config] kinesis_streams: unknown configuration property 'aggregation'. The following properties are allowed: region, stream, time_key, time_key_format, role_arn, endpoint, sts_endpoint, external_id, log_key, and auto_retry_requests.
[2025/03/21 18:43:31] [ help] try the command: /fluent-bit/bin/fluent-bit -o kinesis_streams -h
```
1. How I can use compression and aggregation in kinesis_streams?
2. Is there any documentation that tells what options can be used under extraOutputs?
Contributor guide
Research direction
Start with the kinesis_streams configuration at chart_revision 0.1.35 and the Fluent Bit `-o kinesis_streams -h` output shown in the report. Verify whether extraOutputs is documented or supported, then document the available options and the status or limitation of compression and aggregation. Done means users can determine how to configure these settings or understand that they are unsupported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, helm, yaml
- Domain
- cloud, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100