alibaba / alibaba/Sentinel

MetricSearcher#findOffset eating 5%~15% CPU

Open
#1,409 5 comments 0 reactions 0 assignees View on GitHub
area/performance
Dominant language
Java
Stars
23.1k
Forks
8.1k
PR merge metrics
No merged PRs in 30d

Description

## Issue Description
MetricSearcher findOffset eating 5%~15% CPU,because of DataInputStream#readLong
![image-20200417170907416](https://tva1.sinaimg.cn/large/007S8ZIlgy1gdwvmfm5kvj30h202eweo.jpg)
![image-20200417170544934](https://tva1.sinaimg.cn/large/007S8ZIlgy1gdwviz7jsej30k806bgmw.jpg)

bug report
### Describe what happened
当MetricSearcher工作时,会占用5%~15%的CPU。访问量小的应用没有这种问题,但是部分访问量较高的应用会偶发这个问题,具体访问量是比如一分钟20万次被Sentinel管控的请求。我们有一些用应用单机QPS超过4万,这个问题便会持续出现

When MetricSearcher works, it consumes 5% to 15% of the CPU. There is no such problem with applications with small traffic, but some applications with high traffic will have this problem occasionally. The specific traffic is for example 200,000 requests per minute controlled by Sentinel. We have some applications with a single machine QPS exceeding 40,000, this problem will continue to appear

### Describe what you expected to happen
我们已经在生产环境大规模使用了Sentinel,并且已经基于Sentinel做了内部的版本。官方是否有可能优化这个问题,确实单个线程占用这么高的CPU很难接受。

We have used Sentinel on a large scale in the production environment, and have already made an internal version based on Sentinel. Is it possible for the official to optimize this problem, it is indeed difficult to accept that a single thread occupies such a high CPU.

### How to reproduce it (as minimally and precisely as possible)
分析工具:top+jstack+IBMThread And Monitor Dump Analyzer

### Tell us your environment
Sentinel V1.6.3

Contributor guide

Open the contributing guide

Research direction

Start at MetricSearcher#findOffset and profile it under the reported high-traffic conditions, paying particular attention to DataInputStream#readLong. Compare the profiling results with the reported 5%–15% CPU usage and verify any change against the same workload. Done means the excessive single-thread CPU usage is reduced without changing metric-search behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.