vectordotdev / vectordotdev/vector
The `aws_cloudwatch_logs` sink rate limits should align with streams, not groups
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 146
Description
The AWS CloudWatch PutLogEvents endpoint is rate-limited on a "stream" basis, not a "group" basis:
There is a quota of 5 requests per second per log stream. Additional requests are throttled. This quota can't be changed.
Therefore, it would make more sense to align our request.* options with each stream (not group). My only concern with such a change is that it deviates from our other sinks and might confuse user. In general though, sinks should align with the service they are interfacing with.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the aws_cloudwatch_logs sink and its request.* options, then compare their current grouping with the AWS PutLogEvents stream quota linked in the issue. Done means the rate limits align with individual log streams rather than log groups, while the resulting configuration behavior is clear to users.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- cloud, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100