active activation count for rate limiting is not affected on HTTP response
- Dominant language
- Scala
- Stars
- 6.8k
- Forks
- 1.2k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 2
Description
Creating a load test I received 429 rate limiting responses like `'' has exceeded its throttle limit, Too many concurrent requests in flight (count: 1600, allowed: 1600).`
This was unexpected because:
- the load test operates as 2000 virtual users
- the concurrentInvocations limit for the namespace is 8000
## Steps to reproduce the issue:
1. set namespace limit for concurrentActivations to 8000
2. run a gatling test with blocking requests and 2000 users
3. ? may be affected by using cosmosdb store, which can be slow to store activation records
## Provide the expected results and outputs:
Expect that since number of active requests should be 2000 (or less) at any moment, the 8000 limit should be fine.
## Provide the actual results and outputs:
I received many 429 responses indicating too many concurrent requests.
## Additional information you deem important:
* This may be related to using cosmos to store activations
* I expect similar would happen if log collection is slower than request arrival rate
I'm wondering what is the impact if we move the counter adjustment from`CommonLoadBalancer.processCompletion` into `CommonLoadBalancer.processResult`. From a user perspective, I expect the rate limiting to apply to requests that I have in flight, as opposed to requests that used to be in flight, but are still having logs collected or response records created.
Contributor guide
Research direction
Start with CommonLoadBalancer.processCompletion and processResult, then reproduce the behavior with a Gatling test using blocking requests and 2,000 users. Compare the concurrent activation counter with the HTTP response lifecycle, including the possible effect of a slow Cosmos DB activation store. Done means in-flight requests are counted correctly and the test no longer produces premature 429 responses under the stated namespace limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100