Policy to rate-limit action executions
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.5k
- Forks
- 787
- PR merge metrics
- No merged PRs in 30d
Description
According to https://docs.stackstorm.com/reference/policies.html, we have concurrency and retry policies.
And while concurrency is useful, it's not enough in some cases.
This example comes as a note from SREcon 17 Dublin where we configured demo: Twitter sensor to listen on all #srecon tweets and change LED colors on every tweet.
> Click here for photo
The problem is that sometimes there were too many tweets/per second and we definitely didn't want to change the LED colors with such a high rate. Here limiting executions/per time period could be very useful.
The proposal is to implement rate-limiting policy so we could limit the action executions based on time executions/second, executions/time interval and cancel everything that exceeded the rate limit window.
Proposed policy example
---
name: rate-limit-http-downloads
description: > Run max 3 'core.http' actions/second
and cancel everything that exceeds the limit
enabled: true
resource_ref: core.http
# alternative:
# policy_type: action.limit
policy_type: action.rate_limit
parameters:
# maximum number of executions per time interval window
threshold: 3
# 100ms, 1s, 315s, 15m, 1h
interval: 1s
# cancel actions exceeding the 3 actions/s limit
action: cancel
We'll need additionally to filter/limit the executions by input arguments, same as https://docs.stackstorm.com/reference/policies.html#action-concurrency-attr
I believe this is typical and pragmatic instrument that might be very popular in a wild.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the policy reference at docs.stackstorm.com/reference/policies.html and compare the existing concurrency and retry policy entry points. Define how threshold, interval, cancellation, and input-argument filtering should behave, then identify the relevant policy tests before implementing. Done means the rate-limit example and its boundary cases are covered by tests and documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100