dingo / dingo/api

Multiple Rate Limiting Keys?

Open
#1,002 1 comment 0 reactions 0 assignees View on GitHub
enhancement feature request
Dominant language
PHP
Stars
9.4k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

##### Intro

Hi, :)

If not mistaken, the rate limiting key (RLK) is the string used as an identifier to block/reject the request or not.
I've read the docs and found that I can change rate limiting key (RLK) to, lets say user ID.
The current default RLK is set to client's IP in function `key()` ([In line 245 of Handler.php](https://github.com/dingo/api/blob/ac3b993f426d416c388b14e1513fbe7bcf794ce4/src/Http/RateLimit/Handler.php#L245)), and being called by `cache()` [in line 186](https://github.com/dingo/api/blob/ac3b993f426d416c388b14e1513fbe7bcf794ce4/src/Http/RateLimit/Handler.php#L186).
##### OR gate rate limiting keys

I would like to block both user and IP address. Seems like i need to add another RLK.
It is an OR logic condition.
Meaning once the user i blocked, the user is not able to use the same account on any IP addresses.
And client of the same IP cannot use other user accounts.
##### AND gate rate limiting keys

Appending user ID to the IP address will create an AND condition:
Means only client with the blocked IP using the same user ID will be blocked.
Client with the same IP and different user or User ID with a different client IP will not be blocked.
Since two strings are appended into one, only one RLK is required.
##### Why isn't it implemented?

I wonder why hasn't Dingo API implemented multiple RLKs? I feel it's a widely required condition.
Please do correct me if it's actually a desired feature by most to only block the IP.
If convinced, I might just prefer going with defaults, haha xD
##### Understanding the related lines of code:

Generally we will need to get and set all values in cache based on all the RLKs inserted.
Such as User ID and Client IP.

For example:
When client with User ID `123` and IP `123.123.123.123` submits a requests, the cache will increment the number of requests made by User ID `123` and SEPARATELY, also the number of requests made by the IP `123.123.123.123`

Lets say the throttle limit was set to 10.
Client with combination User ID `123` and IP `123.123.123.123` submits 6 requests.
Now Client with IP `123.123.123.123` changes to User ID `456` and can only send 4 more requests until being blocked. But User ID `456` can still send 6 more requests with other IP address.

I don't think I can add another RLK without changing Dingo API source code.
So having multiple RLKs is considered an enhancement?

Being unsure how to properly accept multiple RLKs, the below are my hypothesis:

Since `cache()` is called by `rateLimitRequest()` multiples times at [line 123 in Handler.php](https://github.com/dingo/api/blob/ac3b993f426d416c388b14e1513fbe7bcf794ce4/src/Http/RateLimit/Handler.php#L123)

Some how an array of RLK strings sound good....
Which means any function related to `$this->cache` will need to run multiple times based on the number of RKLs? Which includes the following functions found in from [line 186](https://github.com/dingo/api/blob/ac3b993f426d416c388b14e1513fbe7bcf794ce4/src/Http/RateLimit/Handler.php#L186):

``` php
protected function cache($key, $value, $minutes) {
$this->cache->add($this->key($key), $value, $minutes);
}

protected function retrieve($key){
return $this->cache->get($this->key($key));
}

protected function increment($key){
$this->cache->increment($this->key($key));
}

protected function forget($key){
$this->cache->forget($this->key($key));
}
```

And.... After that it gets little tricky with `$this->cache->*()`which will require the RLK array.
(wildcard character: '*')

All of `$this->cache->*()` calls `$this->key($key)` that appends several strings including the RLK string using `$this->getRateLimiter()` which will return `$this->limiter`.
**Meaning `$this->limiter` is appended with several strings and stored in cache.**

`$this->limiter` was used to set a single string as the RLK.
To accept multiple limiters, `$this->limiter` will now store array of strings.
That depreciates the previous `$this->key($key)` that can only accept a single string.
##### My proposal of code for multiple Rate Limiting Keys
###### To accept RLK arrays:

Modify `key()` such that it does not retrieve the string from `$this->getRateLimiter()` by itself.
Rather accept the RLK string as a parameter:

``` php
protected function key($key, $singleLimiter)
{
return sprintf('dingo.api.%s.%s.%s', $this->keyPrefix, $key, $singleLimiter);
}
```

and for the functions that uses the functions: `key()` or `$this->cache->*()`

``` php
protected function cache($key, $value, $minutes) {
$limitter = $this->getRateLimiter();
foreach ($limitter as $singleLimtter){
$this->cache->add($this->key($key, $singleLimtter), $value, $minutes);
}
}

protected function retrieve($key){
//used to only return a single value, will now return array of values
// in the same order as the array returned by $this->getRateLimiter()
// return $this->cache->get($this->key($key));
$values = array();
$limitter = $this->getRateLimiter();
foreach ($limitter as $singleLimtter){
array_push($values , $this->cache->get($this->key($key, $singleLimtter)));
}

return $values;
}

protected function increment($key){
$limitter = $this->getRateLimiter();
foreach ($limitter as $singleLimtter){
$this->cache->increment($this->key($key, $singleLimtter));
}
}

protected function forget($key){
$limitter = $this->getRateLimiter();
foreach ($limitter as $singleLimtter){
$this->cache->forget($this->key($key, $singleLimtter));
}
}
```
###### To handle the RLK arrays:

And... `retrieve()` is used by 4 other functions in Handler.php:
This is it when it gets confusing.
`retrieve()` returns array of values because it is mainly used to verify the constraints applied based on either one of the RLKs.

[Line 134](https://github.com/dingo/api/blob/ac3b993f426d416c388b14e1513fbe7bcf794ce4/src/Http/RateLimit/Handler.php#L134)

``` php
protected function prepareCacheStore()
{
//This function is not clearly described, but seems to clear the related caches is the `expires` parameter of the throttle has changed.
// So, if either one of the `expires` values has changed, then clear the caches of all the RLKs.

$expiresValues = $this->retrieve('expires');
foreach ($expiresValues as $expires){
if($expires != $this->throttle->getExpires()){
$this->forget('requests');
$this->forget('expires');
$this->forget('reset');
}
}
}

public function exceededRateLimit()
{
// This function is only used by RateLimit middle ware to determine if the request is blocked, very important.
// It takes only one of the 'requests' value to be larger than $this->throttle->getLimit() and the request is blocked.
if($this->requestWasRateLimited()){
$requestValues = $this->retrieve('requests');
foreach ($requestValues as $request){
if($request > $this->throttle->getLimit()){
return true;
}
}
return false;
} else {
return false;
}

}
```

[Line 302](https://github.com/dingo/api/blob/ac3b993f426d416c388b14e1513fbe7bcf794ce4/src/Http/RateLimit/Handler.php#L302)

``` php
public function getRemainingLimit()
{
// This function is only used to return the remaining amount of requests until limit is reached.
// Which is to return the value that is lowest.
// That is, using the largest 'requests' value.
$remaining = $this->throttle->getLimit() - max($this->retrieve('requests'));
return $remaining > 0 ? $remaining : 0;
}

public function getRateLimitReset()
{
//This function returns the timestamp until the limit expires, so return the latest time in the array.
return max($this->retrieve('reset'));
}

```

The above 2 public functions `getRateLimitReset()` and `getRemainingLimit()` are used only used by the protected function `getHeaders()` in the [RateLimit middleware](https://github.com/dingo/api/blob/ac3b993f426d416c388b14e1513fbe7bcf794ce4/src/Http/Middleware/RateLimit.php) which is meant for returning strings in HTTP headers.
So it is actually fine for the above 2 public functions to return array of values, although not implemented here.
##### Conclusion:

`exceededRateLimit()` is the core function that will determine if the request is blocked or not, by the RateLimit middleware.
The function will retrieve all values in cache based on all the RLKs inserted, and compare all RLKs to see if any of the limits were reached

I'm not certain that this is the best way to implement mutiple RLKs.

I have just forked Dingo API and will now try to implement.
i'm very new to GitHub, I read the "guidelines for contributing"... I think I will have trouble following them.... haha. Please guide me. Thanks!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.