Under concurrent traffic, (possibly) 1. Current limit is not allowed. 2. All the requests are blocked. (Under concurrent traffic, rate limit may not correct, and block all request)
- Dominant language
- Java
- Stars
- 23.1k
- Forks
- 8.1k
- PR merge metrics
- No merged PRs in 30d
Description
## 并发流量下,(可能)1.限流不准/2.拦截所有请求
## Under concurrent traffic,rate limit may not correct, and block all request
### 并发流量, 限流可能不准(放过更多的流量,稳定复现)
中文:
在较大的流量压力(并发)下。限流**可能**不准确。 不准确的点在于, 获取`entry`责任链中, `StatisticSlot` 统计数据, 与`FlowSlot`获取数据,他们两并不是抢占式的。在我的电脑(i5 8400, 16G, MAC10.14.5)上,他们两大约有2ms左右的时间差,而这个时间差足够让后续的线程绕过限流`FlowSlot`插件了。
English:
Under concurrent traffic, rate limit may not correct;
the way for limit is a way of slot chain:
- the slot `FlowSlot` for limiting,
- the slot `StatisticSlot` for statistic;
`FlowSlot` use `StatisticSlot` data. but they do not wait for a lock.
it may cause `FlowSlot` use an old data. And pass a lot of traffic.
### 并发流量下, 部分的请求可能都会被block掉(通过的流量低于设定阈值, 仅出现在线程池调度的情况下)。
中文:
仅在线程池环境下, 该问题复现, 且稳定复现。 我看了很久,没有判断出原因是什么。
现状: 可能在持续长达5s以上的时间, 有流量进入,限流大小不小于1,所有请求被阻断。
English:
only under thread pool, the request may be blocked by sentinel, all the request be blocked, even the limit is larger than 0.
> 这两个问题一般一起出现, 先出现第一个问题, 大约两三秒之后出现第二个问题。 第一个问题导致放过更多的流量, 第二个问题可能导致所有流量全部被block(第二个问题只出现在线程池环境下)。
> the 2 case show together; the first one may cause more than limit traffic; the second may cause no traffic passed(the second case may only appear in thread pool );
### 复现方式(the way to reproduce it)
1. 在主线程中不停的创建子线程, 子线程采用Sentinel推荐的写法(模拟tomcat环境)
(create a lot of child thread, only like this)
```
public static void main(String[] xxx) throws Exception {
XXX x = new XXX();
int j = 99999999;
while (j-- > 0) {
try {
final int av = j;
executor.execute(() -> x.打印一个SystemOut(av));
} catch (Exception E) {
E.printStackTrace();
}
// 当执行休眠的时候, 问题不再复现
/*TimeUnit.MILLISECONDS.sleep(20L);*/
}
System.out.println("shut down");
}
```
2. 使用Sentinel推荐的编码方式
(each thread call the method, like sentinel told us)
3. 在程序运行时, 开启限流。
(open limit on sentinel-dashboard)
### Tell us your environment
mac os x 10.14.5, jdk8u225, eclipse
### Anything else we need to know?
在 限流器 获取当前QPS的时候, 同步一下, 可解决这两个问题。
## 复现问题的全部代码:
```
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import com.alibaba.csp.sentinel.Entry;
import com.alibaba.csp.sentinel.SphU;
import com.alibaba.csp.sentinel.slots.block.RuleConstant;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRule;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRuleManager;
public class ZZZ {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
private static Executor executor = Executors.newFixedThreadPool(50);
private static String pre;
private static int size = 99990000;
public void justPrint(int i) {
try (Entry entry = SphU.entry("XXX.sysGood11")) {
String print = sdf.format(new Date());
if (!print.equalsIgnoreCase(pre)) {
System.out.println();
pre = print;
}
System.out.println(print + "\t" + i);
} catch (Throwable e) {
} finally {
}
}
public static void main(String[] xxx) throws Exception {
ZZZ x = new ZZZ();
int j = size;
initFlowQpsRule();
while (j-- > 0) {
try {
final int p = j;
executor.execute(() -> x.justPrint(p));
/*new Thread(() -> x.justPrint(p)).start()*/
} catch (Exception E) {
E.printStackTrace();
}
/*TimeUnit.MILLISECONDS.sleep(20L);*/
}
System.out.println("shut down");
}
private static void initFlowQpsRule() {
List rules = new ArrayList<>();
FlowRule rule = new FlowRule("XXX.sysGood11");
// set limit qps to 5
rule.setCount(5);
rule.setGrade(RuleConstant.FLOW_GRADE_QPS);
rule.setLimitApp("default");
rules.add(rule);
FlowRuleManager.loadRules(rules);
}
}
```
Contributor guide
Research direction
Start with the reported reproducer using SphU.entry, a fixed thread pool, FlowRuleManager, FlowSlot, and StatisticSlot. Reproduce the QPS limit of 5 under concurrent traffic, then trace how FlowSlot reads StatisticSlot data; done means the concurrency case no longer over-admits traffic or blocks all requests unexpectedly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100