ParamFlowChecker 的QPS 统计时间是每一个请求到来的时刻而非整秒开始时刻
- Dominant language
- Java
- Stars
- 23.1k
- Forks
- 8.1k
- PR merge metrics
- No merged PRs in 30d
Description
## Issue Description
`ParamFlowChecker` 的QPS 统计时间是每一个请求到来的时刻而非整秒开始时刻
在方法`passDefaultLocalCheck(..)` 中,以下代码片段(忽略了无关代码片段)。
```java
...
...
while (true) {
long currentTime = TimeUtil.currentTimeMillis();
AtomicLong lastAddTokenTime = timeCounters.putIfAbsent(value, new AtomicLong(currentTime));
if (lastAddTokenTime == null) {
// Token never added, just replenish the tokens and consume {@code acquireCount} immediately.
tokenCounters.putIfAbsent(value, new AtomicLong(maxCount - acquireCount));
return true;
}
// Calculate the time duration since last token was added.
long passTime = currentTime - lastAddTokenTime.get();
// A simplified token bucket algorithm that will replenish the tokens only when statistic window has passed.
if (passTime > rule.getDurationInSec() * 1000) {
AtomicLong oldQps = tokenCounters.putIfAbsent(value, new AtomicLong(maxCount - acquireCount));
if (oldQps == null) {
// Might not be accurate here.
lastAddTokenTime.set(currentTime);
return true;
} else {
...
...
if (oldQps.compareAndSet(restQps, newQps)) {
lastAddTokenTime.set(currentTime);
return true;
}
Thread.yield();
}
} else {
...
...
}
}
```
从三行代码中可以看出,统计的开始时间并没有做处理,直接使用了当前时间(`currentTime`)
156 行
```java
AtomicLong lastAddTokenTime = timeCounters.putIfAbsent(value, new AtomicLong(currentTime));
```
170 行以及182行
```java
lastAddTokenTime.set(currentTime);
```
这样的统计与限流似乎也没什么问题,但是在`dashboard` 控制台的实时监控那里就会有问题,两者不匹配。
因为控制台中的实时监控那里取的统计数据是通过http 请求,在客户端中由`SendMetricCommandHandler` 实现。
而在客户端这边的读取(`SendMetricCommandHandler`)是从日志文件`metrics.log` 中获取,该文件中的数据是由`MetricWriter` 定时写入的,定时任务时从`ClusterNode` 中取出数据并写入日志文件。实际的统计数据可查看`StatisticNode`,而它统计数据是按整秒([0.000, 1.000))统计的。
这样在实时监控看到的统计数据有时候就会超出我们设置的QPS 非常多。
就是因为这里,实时监控查看到的统计数据是整秒的统计数据。而`ParamFlowChecker` 统计的时间并非整秒,而是首次到来的请求的时间点。
我想问一下,是机制就是这样设计的还是说它就是一个BUG 呢?
Type: *bug report* or *feature request*
bug report
### Describe what happened (or what feature you want)
### Describe what you expected to happen
对关键的三行代码做如下调整,就可以解决该问题。
```java
// 开始时间控制到整秒,与统计时间窗口起始时间一致。
long startTime = currentTime - currentTime % 1000;
AtomicLong lastAddTokenTime = timeCounters.putIfAbsent(value, new AtomicLong(startTime));
...
lastAddTokenTime.set(startTime);
...
lastAddTokenTime.set(startTime);
```
我本来想是否可以使用`StatisticNode` 来进行统计,但是要变更和修改的代码太多了,这样的变更影响最小,如果有问题回溯也会简单一点。
### How to reproduce it (as minimally and precisely as possible)
1.
2.
3.
### Tell us your environment
```xml
com.alibaba.csp
sentinel-parameter-flow-control
1.8.1
```
### Anything else we need to know?
Contributor guide
Research direction
Start in ParamFlowChecker.passDefaultLocalCheck and inspect the timeCounters initialization around line 156 and updates around lines 170 and 182. Read how MetricWriter and StatisticNode define their second-based metrics. Done means ParamFlowChecker uses the same whole-second window so dashboard monitoring and rate-limit statistics align.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100